One Token Can Break LLM Math: ‘Cliff Tokens’ Reveal Hidden Failure Points
Researchers define 'cliff tokens'—individual tokens that cause a sudden drop in reasoning potential, flipping a correct solution into failure. Across seven models and three benchmarks, removing the first cliff token restores performance to perfect scores, while targeting them with specialized DPO training boosts accuracy up to 6.6 points. This pinpoints a new granularity of LLM fragility.