Quantization vs. Training Precision: What 'Smaller' Really Means
Quantization and training precision get treated as the same thing when a vendor says a model is "smaller" — they aren't. Training precision (BF16, increasingly FP8) is the number format baked in while a model learns; post-training quantization (INT8, INT4) compresses an already-trained model afterward for cheaper serving. The distinction matters more now that FP8 training and aggressive quantization have become standard levers for cutting inference cost. For PMs, knowing which one you're buying changes what quality tradeoff you should actually expect — and what you should test before signing off.