Maglev gives Transformers a fixed-size memory for faster long-context training
Maglev pairs a full‑attention prefiller with a recurrent decoder that uses sliding‑window attention and a compressed memory slot, letting inference run on the lightweight decoder alone. The memory‑consistency loss boosts validation loss and downstream pre‑training benchmarks while cutting parameter memory versus competing recurrent transformers.