GPU idle most of the time serving diffusion LLMs: 24% compute, the rest is dispatch
Masked diffusion LLMs promise faster generation by denoising many tokens at once, but nobody had measured them under real concurrent load. On an H200 running LLaDA-8B, this paper finds only 24% of wall-clock time is GPU compute; the rest is CPU-side dispatch overhead that batching amortizes, giving 16x throughput at batch 16. It also shows denoising steps land on 11 discrete levels no feature predicts, so AR-serving assumptions quietly break. The lesson: serving dLLMs needs parallelism inside each denoising step, not just across requests.