A faculty-and-graduate-researcher audit of six physics benchmarks found wrong answer keys, ambiguous prompts, and grader bugs behind many apparent model failures. After fixing or removing bad questions, GPT-5.6-Sol rose from 47.3% to 78.7% on HLE-Physics, and reached 94.4% pass@4 on the retained CritPt set. The uncomfortable takeaway: closed-ended physics tests may now be nearing saturation, so evaluation needs harder, expert-validated problems.
Mathematics used to rely on scarce, difficult theorems as a rough signal that someone had developed real understanding. Bryna Kra argues that AI is breaking that signal: models can now produce polished proofs, including attacks on long-standing problems such as the Nivat conjecture, faster than experts can verify and absorb them. The bottleneck is shifting from discovery to deciding which machine-made results deserve trust and attention.
Public coding benchmarks can make agents look ready for the job. Real-SWE puts them inside licensed, private enterprise codebases, where tasks touch billing, taxes, customer migrations, and company-specific conventions; the top model-and-harness combination resolved only 38.8% of tasks. That makes context, integration, and business rules the bottleneck, not just code generation, and it is a tougher baseline for anyone deploying agents at work.
Astra solves randomized Boolean-logic problems without visible chain-of-thought, then gets markedly better when given meaningless filler tokens to process. Tracking its per-variable confidence suggests the scores oscillate before converging toward posterior probabilities, much like iterative belief propagation. If the result holds up, “no CoT” does not mean “no multi-step computation”: useful reasoning may be happening in a hidden continuous state.
Most models spend a fixed amount of computation on each token. Recurrent Looped Transformer instead carries decoder state and layerwise attention caches forward, so a 48-layer decoder traverses 48 blocks per token and builds a temporal path of 48t blocks across a sequence. The design is unusually ambitious, but the page is explicit: reasoning gains, hardware efficiency, and RL scaling are still unproven.