Transformer 'Massive Activations' Survive Dedicated Answer Channel, Proving They’re Functional
A new Ledger Residuals architecture separates the residual stream into a writable scratch space and a read‑only accumulator. Experiments on 160‑M and 290‑M language models show the characteristic start‑token activation re‑emerges in the protected channel, meaning the phenomenon is not an artifact but a robust functional feature. This impacts quantization and pruning strategies.