Circuit analysis across 46 tasks shows LLMs spontaneously organize into specialized networks for language, formal reasoning, social cognition, and physics, mirroring the human brain's functional modularity. Because these models were optimized by a totally different process, the convergence hints modularity is a fundamental property of intelligence, not a biological accident. That raises concrete questions about interpretability and how we should architect future systems.
A DeepMind position paper (ICML 2026) argues LLMs master induction and deduction but lack abduction: the leap that turns sparse observation into a new explanatory hypothesis, like Einstein's equivalence principle. The fix isn't more scale, but physically grounded multimodal world models to bridge the gap.
SSOG-Attention ditches query-key dot products for a learned field of separable Gaussian atoms, each head just two 1D filter passes instead of an N×N matrix. It hits 65.3% on ImageNet-1K and 70.4% on CIFAR-100 (dot baseline: 53.1%), with free zero-shot resolution transfer. The open question: what long-range recall trades away for the near-linear compute?