Choosing a numeric precision (FP8, BF16, FP16)

BF16 is the safe default for training on modern GPUs — wide dynamic range, good stability. FP16 is common for inference. FP8 (Hopper/Blackwell, CDNA3) roughly doubles throughput again and halves memory, and is increasingly used for both inference and training with care.

Not every part supports FP8 natively — Ampere (A100) does not. Check the catalog's precision fingerprint before assuming FP8 speedups.

For inference, quantizing to FP8/INT8 is one of the biggest levers on cost and latency; validate quality on your eval set before rolling out.