Grokking Has a Measurable Trigger, and Data Beats Width
Grokking may look mysterious, but this study finds a measurable control panel: across 384 MLP configurations, doubling the data cut generalization time by roughly 4×, while doubling width improved it only about 1.2×. The sharpest switch was weight decay, with values above 1.0 separating grokking from runs that never made the transition. That gives researchers a way to predict and tune the delayed jump from memorization to generalization.