A KV-cache fix lets looped LMs skip 30% of computation
Looped language models can now cut up to 30% of FLOPs and KV-cache memory while retaining full-depth performance, making their promised token-by-token compute savings practical. The paper’s “best-available” caching keeps the usable intermediate state instead of forcing every loop iteration to run, while a small change to the early-exit objective lets easy and hard tokens stop at different depths. Tests cover Ouro models and smaller looped LMs trained from scratch.