Fused-kernel attention breaks the memory wall: 11.6x faster long-context decoding
Long-context decoding stalls on memory bandwidth, not compute. Faster Flash Decoding fuses the attention selector and compute into one kernel, replacing metadata indices with low-bit scanning plus a top-delta filter that drops blocks adaptively, no retraining required. The payoff: up to 11.6x kernel speedups, 2.37x end-to-end throughput, and 256K context without losing accuracy on RULER or LongBench. Attention sparsity just became practical for production inference.