Kog AI’s engine delivers 3,000-token-per-second LLM decoding on standard GPUs
Kog AI unveiled a tech preview of its Kog Inference Engine, achieving up to 3,000 output tokens per second on a cluster of eight AMD MI300X GPUs (2,100 t/s on NVIDIA H200). The speed‑focused stack co‑optimizes model architecture, runtime, and low‑level kernels, making real‑time LLM inference feasible on existing datacenter GPUs.