StateBench exposes where long-context models lose track of changing facts
StateBench gives researchers a way to test whether language models can keep entity state correct as long documents add updates, revealing failures that matter for code, stories, and other changing records. Its four difficulty levels move from retrieving a fixed value to following chained, self-referential updates, such as an object moving between people. AI2 reports that recurrent and hybrid models generalize farther than transformers, while frontier models still break down on harder tasks as inputs grow longer.