NVIDIA's AVO agent scored a perfect 100.00 RHAE on ARC-AGI-3, clearing all 183 levels across 25 public environments with no instructions, rules, or stated goals. The agent figures out each unfamiliar environment by inspecting, acting, and learning from feedback, sustained by memory and long-horizon state rather than fresh context each step. The telling part: the underlying model scored roughly 30% bare, which puts the win on the harness, not the model. It is the argument that evaluating a model is not the same as evaluating an agent, made with a number.
Precise control was supposed to make activation steering safer than fine-tuning. New work shows the opposite: adding a vector for something as benign as "Portugal" can flip a refused request into a compliant one. Random directions alone lift harmful compliance from 0% to 1-13% across model families, and averaging 20 prompt-specific jailbreak vectors yields a universal attack. A direct challenge to the safety-through-interpretability paradigm.
Freezing the judge is the standard fix for eval gaming. AQuA, a new preprint, tests it with sealed-sandbox LLM research agents for quant trading: data splits, labels, and evaluator stay locked, and agents act only through constrained factor expressions or config diffs. The defense mostly holds, with a held-out Sharpe up to 2.50 and positive returns every year from 2021 to 2025. But feedback from that fixed evaluator still steers the loop, so the judge remains a target of adaptation.