What is cuBLASLt?

A descriptor-based matrix multiplication interface that exposes algorithm heuristics, workspace choices, layouts and fused epilogues beyond traditional cuBLAS GEMM calls.

cuBLASLt turns GEMM into a described operation rather than one fixed function signature. You create descriptors for the operation and matrix layouts, request heuristic algorithms under a workspace limit, then call cublasLtMatmul. This makes row-major layouts, mixed-precision compute types and FP8 paths easier to express than in traditional cuBLAS.

An epilogue applies work such as bias while the output tile is still inside the chosen implementation. That can remove a global-memory round trip and a launch, the same mechanism as kernel fusion. The saving is not necessarily equal to the standalone bias kernel's time because enabling an epilogue may select a different GEMM algorithm.

Heuristic order is not a performance guarantee. Candidate zero is the library's first recommendation, not proof of the fastest result for every size. Time all returned candidates using the same descriptors, workspace ceiling and correctness gate before persisting an algorithm choice.

Measured

On a Tesla T4 (driver 580.173.02, CUDA 12.6), day 81 measured the 2048-square FP32 Lt path plus a separate bias kernel at 6.1812 ms. The fused bias epilogue took 4.7476 ms, saving 1.4336 ms or 23.2 percent. In the separate eight-candidate search, heuristic candidate 0 took 3.8400 ms while candidate 7 took 3.5681 ms, 7.6 percent faster. At 512, candidate 0 was fastest. All candidates used at most the stated 32 MiB workspace and the selected paths matched the CPU reference.

Related terms

• cuBLAS
• tensor core
• FP8
• kernel fusion
• mixed precision

Where you meet this

• Day 81, cuBLAS and cuBLASLt, which measured the epilogue and heuristic search.
• Day 48, kernel fusion, where the same deleted-byte model is built by hand.
• How to set up CUDA, because cuBLASLt is a separate library link inside the toolkit.

Sources

• cuBLAS documentation, cuBLASLt descriptors, heuristics and epilogues: https://docs.nvidia.com/cuda/cublas/index.html (checked 2026-09-01)

Byline

Written by: pending. Reviewed by: pending. Written on: pending. Last checked on: pending. Verified numbers were captured on 2026-09-02; publication still requires named author and reviewer sign-off.