vLLM adds portable layers without giving up H100 speed
vLLM’s new hardware-agnostic layers let model authors target out-of-tree accelerators, older GPUs, and unusual models without accepting a major performance penalty on NVIDIA H100s. They preserve fullgraph torch.compile support while separating portable model logic from hardware-specific kernels and definitions. Across three recent models, token throughput was within 3.4% of vLLM’s native implementation on H100 GPUs.