2:4 structured sparsity

2:4 structured sparsity keeps exactly two values in each consecutive group of four along a matrix's supported dimension.

The fixed pattern gives hardware enough structure to compress the matrix and skip known zero positions without storing an arbitrary sparse index for every value. On supported NVIDIA tensor cores, libraries such as cuSPARSELt can use the pattern for sparse matrix multiplication. Pruning a dense matrix into the pattern changes its values, so performance and numerical quality must be evaluated separately.

Day 98 applies magnitude pruning: within every four-value group it keeps the two entries with the largest absolute values. The Tesla T4 has compute capability 7.5 and does not provide the Ampere-generation 2:4 sparse tensor-core path, so the lesson records pruning accuracy only. It does not report a cuSPARSELt speedup or imply that zeros alone accelerate the dense CUDA-core kernel.

Measured

At matrix size 2048, the pruned result had maximum absolute error 4.120e+01 and RMS error 2.865. The dense per-column INT8 result on the same workload had maximum error 8.311 and RMS error 0.2468, making the pruned RMS error about 11.6x larger.

No sparse-kernel timing or profiler artifact was captured because the verification GPU lacks the required sparse tensor-core capability. The intended cuSPARSELt speedup comparison therefore remains unmeasured rather than being inferred from the number of zeros.

Diagram: Each consecutive group of four weights retains the two largest magnitudes, while the other two positions become zero before sparse tensor-core multiplication.

Related terms

• INT8 quantization
• MX block scaling
• Tensor core
• cuSPARSE
• Mixed precision

Where you meet this

• Day 98, quantization and sparsity, which contrasts the accuracy cost of 2:4 magnitude pruning with dense low-precision storage and computation.
• Day 82, CUDA math libraries, which measures general sparse matrix-vector multiplication through cuSPARSE rather than the structured tensor-core path.

Sources

• NVIDIA cuSPARSELt documentation: https://docs.nvidia.com/cuda/cusparselt/ (checked 2026-09-01)
• CUDA compute-capability appendix, 2:4 structured-sparsity support: https://docs.nvidia.com/cuda/cuda-programming-guide/05-appendices/compute-capabilities.html (checked 2026-08-29)

Byline

Written by: pending. Reviewed by: pending. Written on: pending. Last checked on: pending. Verified numbers were captured on 2026-09-02; publication still requires named author and reviewer sign-off.