A Nucleation Rate Law Predicts When Language Model Capabilities Ignite and Fade
The authors model capability emergence in transformers as a driven nucleation event: a circuit must align perfectly in a single stochastic attempt, otherwise nothing accrues. Their rate law accurately predicts emergence timing, irreversible plasticity loss, and shows that re‑initializing query‑key slices can revive stalled capabilities. This provides concrete levers for controlling training dynamics.