Why load balancers break for real-time AI and what to track instead
Google explains why standard request-based load balancing collapses for real-time agents: long-lived bidirectional streams carry audio buffers, transcripts, tool calls, and context, so QPS and CPU miss the real load. Balance both signals by adding active-session counts to route work sensibly.