KV cache compression risks quantified and mitigated for long‑sequence transformers
The paper derives minimax risk bounds for KV cache compression, shows when aggressive compression silently degrades output, and introduces risk metrics plus a practical algorithm that meets these guarantees and improves LongBench results. This matters because KV cache compression is widely used to speed long‑sequence inference, and without proper risk assessment models can silently fail.