Editable KV Cache Cuts LLM Prompt Latency up to 15×
The paper shows KV caches can be edited and composed during prefill, so only changed fields are overwritten while the rest is reused. This yields up to 14.9× lower latency and 53‑398× faster time‑to‑first‑token in vLLM benchmarks without hurting model outputs.