Inference speedups fail when quality is the constraint
A new Pareto atlas tests 54 inference configurations across Qwen2.5-7B, three GPUs, and multiple quantization and caching strategies. FP8 weights preserve 99.4% of baseline GSM8K accuracy while cutting latency to 0.61–0.65×, but a naive FP8 KV cache gets zero answers correct. The practical takeaway: benchmark the whole quality-cost-latency frontier, because the fastest configuration may be unusable and combined methods often win.