Inference is now a storage workload: GPUs idle while state piles up in flash
Long-context sessions, agents, and RAG churn out KV caches and embeddings that are cheaper to store than recompute, so inference is becoming a state-management problem. Cached-token discounts of 90-99% prove the economics, and flash, not HBM, is where the battle moves. That redraws the memory-storage boundary in AI infrastructure.