What is mixed precision in CUDA?

Computing with narrow input or storage formats while retaining a wider accumulator, so bandwidth and arithmetic throughput improve without paying narrow-format error at every sum.

Mixed precision makes two decisions separately: how values are stored and multiplied, and how their running sum is represented. FP16 inputs halve bytes relative to FP32 and can feed tensor cores. Keeping the accumulator in FP32 prevents every partial sum from being rounded back to FP16.

This matters most in reductions such as matrix multiplication. Input conversion introduces one rounding error per stored value. Those errors often partly cancel across K products. A narrow accumulator adds another rounding at the scale of the growing partial sum on every iteration, so its error can grow much faster. Wider accumulation cannot recover bits already lost from the input, but it removes that second source.

The term does not mean “turn on half precision everywhere.” BF16, TF32 and FP8 make different range and precision trades and have different hardware floors. A correct experiment names input type, accumulator type, output type and compute capability. It also checks non-finite values before applying a tolerance.

Measured

On a Tesla T4 (driver 580.173.02, CUDA 12.6), day 71 ran identical tiled matmuls with FP16 storage and either FP32 or FP16 accumulation. At K=256, maximum errors were 2.595e-03 and 2.950e-02, a ratio of 11.4. At K=1024 the ratio reached 51.3. At K=2048, FP32 accumulation produced 2.016e-03 maximum error while FP16 accumulation produced 5.078e-01, a 252x ratio. Both paths passed their stated gates; the ratio reports the accuracy price of the accumulator rather than declaring the lower-precision path incorrect.

Related terms

• accumulator type
• FP16
• BF16
• TF32
• tensor core
• WMMA

Where you meet this

• Day 71, mixed precision, which owns the measurements above.
• Day 72, WMMA, where input and accumulator types become fragment declarations.
• Colab setup, where the sm_75 FP16 path can be reproduced.

Sources

• NVIDIA, Mixed Precision Training, for wider accumulation and loss-scaling practice: https://docs.nvidia.com/deeplearning/performance/mixed-precision-training/index.html (checked 2026-09-01)
• PTX ISA, alternate floating-point formats and matrix instruction types: https://docs.nvidia.com/cuda/parallel-thread-execution/index.html (checked 2026-09-01)

Byline

Written by: pending. Reviewed by: pending. Written on: pending. Last checked on: pending. Verified numbers were captured on 2026-09-02; publication still requires named author and reviewer sign-off.