JetSpec shatters speculative decoding limits with up to 9.6× faster local LLM inference
JetSpec introduces causal parallel tree drafting, letting a frozen target model verify an entire speculative tree in one pass. On the MATH‑500 benchmark it delivers a 9.64× end‑to‑end speedup and sustains ~1000 tokens‑per‑second on a single B200 GPU, unlocking real‑time local LLM serving.