vLLM 0.28 goes all-in on Kimi-K3 and DeepSeek V4 performance
Open models keep leapfrogging, and the engine that serves them just sprinted to catch up. vLLM 0.28.0 (584 commits, 270 contributors) is a full-stack push for Kimi-K3: fused decode kernels, an adaptive speculative token budget, and shared-expert sharding that frees ~17 GiB per GPU. DeepSeek V4's sparse MLA now runs end-to-end across decode, MTP, and DSpark speculative decoding, with ROCm support for both. Plus disk-tiered KV cache offloading and a Rust gRPC frontend.