4-bit quantization keeps coding-agent accuracy but changes how agents reach answers
Quantization stores model weights in fewer bits to cut cost. Comparing BF16 with 4-bit AWQ and GPTQ versions of Qwen2.5-Coder-Instruct (7B, 14B, 32B) and Devstral-Small-2507 on OJBench, the authors find no statistically significant accuracy loss. Behavior shifts underneath: AWQ cuts Qwen-7B's one-shot solves from 17 to 0 while loop-based solves rise from 10 to 25. Devstral's failures move from runtime errors toward wrong answers and timeouts.