Hierarchical Global Attention lets pretrained LLMs run 64K tokens without retraining
HGA replaces dense causal attention with a two‑level routing scheme that keeps the original QKV/O weights unchanged, so any checkpoint can be patched and run long contexts. On a RTX 5090 it runs a 30B model at 64K tokens using only a tiny routed working set, with negligible quality loss.