When extra reasoning derails LLMs: fragile correctness revealed
A study of Gemma 4‑12b‑it’s chain‑of‑thought outputs shows models often reach a correct answer before a later reasoning step flips them wrong—a phenomenon the authors call “fragile correctness.” Roughly 15% of MMLU‑pro and GPQA‑diamond questions exhibit this, and simple linear probes can recover about 1% overall accuracy, with larger gains on the affected cases.