LodeHQ
AI news for people who shipFree, by email

AI · github.com · Monday, 22 June 2026

llama.cpp adds multi‑layer flash MTP for Step 3.5/3.7 speculative decoding

The latest llama.cpp pull‑request brings multi‑layer MTP (Multi‑Token Prediction) support, enabling Step 3.5/3.7 flash MTP. This lets the model run speculative decoding across several layers, boosting inference speed and closing the gap with vLLM‑style kernels.

Read the source: github.com ↗In the 22 June issue →