Agent-built vLLM path pushes Kimi K3 past 840 tokens per second
vLLM’s agent-assisted Kimi K3 decode path delivers over 840 tokens/s at batch 1 on 16 NVIDIA GB300 GPUs, giving teams a faster way to serve the model and up to 1.8× higher P90 interactivity on the AgentX workload. It replaces much of the execution with custom CuTe DSL kernels that fuse attention, mixture-of-experts, and communication. vLLM also proposes isolating these optimizations and requiring reproducible correctness and performance checks before review.