Ant Group and vLLM's open Humming kernels raised MoE serving throughput 14%–43%
Humming is an open-source library of GPU kernels, the small programs that do a model's math, for quantized models that store numbers in fewer bits to save memory. One library covers many integer and floating-point formats on NVIDIA GPUs from Turing onward. In a vLLM test, switching a MoE model from Marlin W4A16 to Humming W4A8 raised end-to-end throughput by about 14%–43% and cut mean time to first token by about 25%–32%.