Tiny-vLLM: High-Performance C++/CUDA LLM Inference Engine
Tiny-vLLM is an open-source LLM inference server written from scratch in C++ and CUDA, offering low-latency prefilling, decoding, KV-cache, and advanced techniques like FlashAttention-style online softmax and PagedAttention. It includes a step-by-step tutorial, making it both a production-ready tool and a learning resource for GPU-accelerated AI.