Quantization Shrinks Decision Margins, Multiplying LLM Failures
The authors show that quantizing LLMs cuts decision margins by a factor that drops dramatically as bits shrink—4‑bit reduces margins to 86 % of original, 3‑bit to 33 %, 2‑bit wipes them out. This multiplicative shrinkage lets compressed models silently stop calling tools and halve safety refusals while benchmark scores stay flat, forcing a rethink of how we assess quantization risk.