Low-bit quantization can double token usage in reasoning models, killing speed gains
Post‑training INT3/INT4 quantization keeps accuracy but forces reasoning models to generate longer chains of thought, inflating token counts by up to 2×. The extra compute erodes the expected latency benefits, meaning low‑bit inference may not speed up real‑world deployments.