GitHub Copilot’s agentic coding overloads LLM caches, demanding new infra
Researchers sampled 13 million GitHub Copilot sessions and found that AI coding agents run sparse user turns packed with chained LLM calls and tool executions. Cache hit rates stay near 90 % inside a turn but collapse to 55 % across turns, while idle periods dominate runtime. The pattern forces a rethink of LLM‑serving architectures and prompts proactive idle‑time prediction.