vLLM 0.32.0 halves KV memory and boosts short-request serving
vLLM 0.32.0 gives serving teams a lower-memory path for GLM-5.x and 43-85% more requests per second for short requests when they run multiple API workers. GLM-5.x now defaults to an FP8 KV cache, halving KV memory per rank, while the least-loaded worker handles each connection. The release also adds NVFP4 caches, GPU n-gram speculation, and fixes three security issues.