BuggyMLE shows that coding agents can look strong on isolated tasks yet fail on real machine-learning repositories, where fixing one bug can break another file. That gap matters if you plan to trust agents with production code.
Research
BuggyMLE gives researchers a harder, repository-level test for ML coding agents, where the best solve rate falls from 95% to 9% across its easiest and hardest tiers. It injects controlled bugs into working machine-learning repositories, producing more than 6,600 variants from 222 source repos. Training on three difficulty levels also pushed a Qwen3.5-9B model’s SWE-bench Lite resolve rate from 24% to 33%.
A new result proves the weaker sample compression conjecture for every binary concept class with VC dimension d, replacing the previous exponential-in-d bound with a scheme of size O(d² log² d). In plain terms, a consistent predictor can be represented using a much smaller part of its training sample, supporting tighter generalization analysis. The stronger goal, a linear O(d) scheme, remains open.
Researchers found a way to audit and alter GraphCast’s internal weather dynamics, reducing intensity error by a median 51% across seven storms when they amplified features associated with convection. Sparse autoencoders expose recurring internal patterns, then researchers turn those features up or down during the forecast rollout. The tests also show that a feature tied to the model’s computational mesh can be causally important even without an obvious physical meaning.
InflEns links how mixture-of-experts data is allocated during training to which expert handles each prompt, so specialization can be inspected before deployment and harmful behavior traced to one expert. Its location game places experts in a prompt-trait space, then reuses those positions for routing without training a separate gate. On seven safety benchmarks, the highest-share expert had the lowest held-out NLL on about 90% of prompts, versus about 15% after shuffling allocations; retraining one expert cut HarmBench attack success from 0.51 to 0.02 while preserving instruction following.
A reference-free reward model lets RL train a 1.7B language model nearly as well as an answer key, raising MATH500 accuracy from 72.2% to 76.8%. It works by asking an LLM a typed question about each response, such as whether the final answer is correct, then using its calibrated probability rather than a raw score. The reward's calibration error was 0.074, versus 0.141 for a softmax judge, but it became overconfident on problems the base model mostly failed.
A new method gives time-series foundation models extra inference compute while preserving causal forecasting, so they can improve predictions without peeking at future values. Backtest-Gated Augmentation Averaging creates candidate views of a series's history, tests them on rolling historical forecasts, drops views that lose to the original context, and averages the survivors. Across three benchmarks, it cuts MASE by 0.5-5.9%; on BOOM, a 313M model with the method beats 1B and 2.5B models in its family.
Pro² Agent gives autonomous AI research agents a structured record of what blocks success, helping them escape solution paths that code-only iteration can miss. It analyzes resources and constraints, tests candidate fixes, and updates which problems matter most; on complete MLE-Bench, it matches the best reported 24-hour medal rate at 70.7% and reaches 76.0% with 48 hours. It also produces reusable research notes.
A new agent lets self-driving laboratories choose measurements and interpret results while keeping every state-changing action authorized, validated, and auditable. Tested on an LC-MS instrument during an oxidant-screening campaign, it matched scripted peak integration and chemists confirmed its spectral interpretations and failure assignments. In a replay simulation, it found the highest-yielding condition at about its seventh injection, versus 14.5 for random ordering, and found it in every campaign with eight or more injections.
Google Quantum AI shows why a quantum computer can beat every classical strategy in a simple game: classical players can win at most 8 of 9 rounds, while quantum correlations let them win every round in principle. The game gives Alice a row and Bob a column from a 3×3 grid; their measurements agree at the intersection even though no fixed table can satisfy both sets of rules. The experiment scales the test to systems of up to 105 qubits.
A new benchmark gives researchers a way to test whether conversational LLMs strengthen users’ parasocial attachment over multiple turns, rather than only measuring one reply. CoMPASS gradually escalates anthropomorphism, attachment, and dependency across four user personas, then scores seven signals against a matched control. In its evaluation, six of eight frontier models showed significantly more overall reinforcement when all three dimensions were escalated; provider differences were larger than differences within a provider.
AI agents still fail nearly half of EverydayBench’s live-web tasks, so strong demos do not yet guarantee usable results for product research, trip planning, or artifact creation. The benchmark tests 77 human-authored tasks across eight domains, checking hard constraints, verifying claims against webpages retrieved during evaluation, and judging the final deliverable. Among 23 agent-model configurations, Claude Opus 5.5 and Claude Fable 5.1 tie for the highest strict pass rate at 54.55%.
No algorithm can guarantee that a language-model agent paired with a general-purpose computer will avoid a dangerous program, so guardrails, blacklists, whitelists, and certifications can only provide imperfect safety. The paper shows that bounded agents are technically decidable but still PSPACE-complete to analyze. It also finds that separately safe agents can become unsafe when combined, breaking safety checks based on components alone.
A new label-free method lets researchers tune model-merge coefficients from unlabeled probes, avoiding repeated construction and evaluation of merged networks. It first measures how one task vector changes another task’s outputs at the receiver model, then builds a simple surrogate that answers new coefficient and task-subset queries in seconds. LR-Merge improved every tested base merger and matched exact teacher matching, whose optimization took roughly three times longer to converge.
A proposed memory system lets an LLM retain stable user facts, update changed beliefs, and let temporary information expire without relying only on retrieval. It stores facts in a fixed-capacity LoRA adapter, then uses a lightweight controller to decide what to remember, how deeply, and for how long. On long-horizon user trajectories, the policy reached 86% of oracle performance while testing both recall during validity and abstention after expiry.
SailSearch gives algorithm researchers a human-guided workflow with two language-model agents, and reports new CPU-only solutions for 18 of 33 previously open Market Split instances. It splits computation between enumerating candidates and reducing lattice bases, solving 130 to 133 of 156 benchmark instances per seed within 24 hours on two CPU cores. On seven maximum-independent-set tests, existing tested solvers still finished 5 to 15 vertices below certified optima.