LodeHQ
AI news for people who shipFree, by email

AI · Friday, 20 November 2026 · Issue 176 · 18 stories

BuggyMLE cuts AI coding success from 95% to 9%

BuggyMLE shows that coding agents can look strong on isolated tasks yet fail on real machine-learning repositories, where fixing one bug can break another file. That gap matters if you plan to trust agents with production code.

Research

BuggyMLE makes ML coding agents face real multi-file failures

BuggyMLE gives researchers a harder, repository-level test for ML coding agents, where the best solve rate falls from 95% to 9% across its easiest and hardest tiers. It injects controlled bugs into working machine-learning repositories, producing more than 6,600 variants from 222 source repos. Training on three difficulty levels also pushed a Qwen3.5-9B model’s SWE-bench Lite resolve rate from 24% to 33%.

arxiv.org ↗2 min read at source

A new bound makes sample compression polynomial for every VC class

A new result proves the weaker sample compression conjecture for every binary concept class with VC dimension d, replacing the previous exponential-in-d bound with a scheme of size O(d² log² d). In plain terms, a consistent predictor can be represented using a much smaller part of its training sample, supporting tighter generalization analysis. The stronger goal, a linear O(d) scheme, remains open.

arxiv.org ↗1 min read at source

GraphCast’s hidden features can improve cyclone forecasts by 51%

Researchers found a way to audit and alter GraphCast’s internal weather dynamics, reducing intensity error by a median 51% across seven storms when they amplified features associated with convection. Sparse autoencoders expose recurring internal patterns, then researchers turn those features up or down during the forecast rollout. The tests also show that a feature tied to the model’s computational mesh can be causally important even without an obvious physical meaning.

arxiv.org ↗1 min read at source

InflEns makes MoE routing predictable before training

InflEns links how mixture-of-experts data is allocated during training to which expert handles each prompt, so specialization can be inspected before deployment and harmful behavior traced to one expert. Its location game places experts in a prompt-trait space, then reuses those positions for routing without training a separate gate. On seven safety benchmarks, the highest-share expert had the lowest held-out NLL on about 90% of prompts, versus about 15% after shuffling allocations; retraining one expert cut HarmBench attack success from 0.51 to 0.02 while preserving instruction following.

arxiv.org ↗2 min read at source

A calibrated judge raises MATH500 accuracy by 4.65 points

A reference-free reward model lets RL train a 1.7B language model nearly as well as an answer key, raising MATH500 accuracy from 72.2% to 76.8%. It works by asking an LLM a typed question about each response, such as whether the final answer is correct, then using its calibrated probability rather than a raw score. The reward's calibration error was 0.074, versus 0.141 for a softmax judge, but it became overconfident on problems the base model mostly failed.

arxiv.org ↗2 min read at source

Backtesting lets forecasters spend compute without seeing the future

A new method gives time-series foundation models extra inference compute while preserving causal forecasting, so they can improve predictions without peeking at future values. Backtest-Gated Augmentation Averaging creates candidate views of a series's history, tests them on rolling historical forecasts, drops views that lose to the original context, and averages the survivors. Across three benchmarks, it cuts MASE by 0.5-5.9%; on BOOM, a 313M model with the method beats 1B and 2.5B models in its family.

arxiv.org ↗1 min read at source

Pro² Agent lifts AI research by tracking problems, not just code

Pro² Agent gives autonomous AI research agents a structured record of what blocks success, helping them escape solution paths that code-only iteration can miss. It analyzes resources and constraints, tests candidate fixes, and updates which problems matter most; on complete MLE-Bench, it matches the best reported 24-hour medal rate at 70.7% and reaches 76.0% with 48 hours. It also produces reusable research notes.

arxiv.org ↗1 min read at source

Bounded AI agent cuts self-driving lab search to about seven injections

A new agent lets self-driving laboratories choose measurements and interpret results while keeping every state-changing action authorized, validated, and auditable. Tested on an LC-MS instrument during an oxidant-screening campaign, it matched scripted peak integration and chemists confirmed its spectral interpretations and failure assignments. In a replay simulation, it found the highest-yielding condition at about its seventh injection, versus 14.5 for random ordering, and found it in every campaign with eight or more injections.

arxiv.org ↗2 min read at source

Quantum contextuality turns an impossible grid into a winnable game

Google Quantum AI shows why a quantum computer can beat every classical strategy in a simple game: classical players can win at most 8 of 9 rounds, while quantum correlations let them win every round in principle. The game gives Alice a row and Bob a column from a 3×3 grid; their measurements agree at the intersection even though no fixed table can satisfy both sets of rules. The experiment scales the test to systems of up to 105 qubits.

research.google ↗8 min read at source

CoMPASS shows six of eight models reinforce parasocial attachment

A new benchmark gives researchers a way to test whether conversational LLMs strengthen users’ parasocial attachment over multiple turns, rather than only measuring one reply. CoMPASS gradually escalates anthropomorphism, attachment, and dependency across four user personas, then scores seven signals against a matched control. In its evaluation, six of eight frontier models showed significantly more overall reinforcement when all three dimensions were escalated; provider differences were larger than differences within a provider.

arxiv.org ↗1 min read at source

EverydayBench finds top agents pass only 54.55% of real-world tasks

AI agents still fail nearly half of EverydayBench’s live-web tasks, so strong demos do not yet guarantee usable results for product research, trip planning, or artifact creation. The benchmark tests 77 human-authored tasks across eight domains, checking hard constraints, verifying claims against webpages retrieved during evaluation, and judging the final deliverable. Among 23 agent-model configurations, Claude Opus 5.5 and Claude Fable 5.1 tie for the highest strict pass rate at 54.55%.

arxiv.org ↗2 min read at source

General-purpose agents cannot be fully safety-verified

No algorithm can guarantee that a language-model agent paired with a general-purpose computer will avoid a dangerous program, so guardrails, blacklists, whitelists, and certifications can only provide imperfect safety. The paper shows that bounded agents are technically decidable but still PSPACE-complete to analyze. It also finds that separately safe agents can become unsafe when combined, breaking safety checks based on components alone.

arxiv.org ↗1 min read at source

LR-Merge tunes model combinations without rebuilding every candidate

A new label-free method lets researchers tune model-merge coefficients from unlabeled probes, avoiding repeated construction and evaluation of merged networks. It first measures how one task vector changes another task’s outputs at the receiver model, then builds a simple surrogate that answers new coefficient and task-subset queries in seconds. LR-Merge improved every tested base merger and matched exact teacher matching, whose optimization took roughly three times longer to converge.

arxiv.org ↗2 min read at source

A LoRA memory system lets stale user facts fade inside model weights

A proposed memory system lets an LLM retain stable user facts, update changed beliefs, and let temporary information expire without relying only on retrieval. It stores facts in a fixed-capacity LoRA adapter, then uses a lightweight controller to decide what to remember, how deeply, and for how long. On long-horizon user trajectories, the policy reached 86% of oracle performance while testing both recall during validity and abstention after expiry.

arxiv.org ↗2 min read at source

SailSearch finds new CPU-only solutions to 18 open instances

SailSearch gives algorithm researchers a human-guided workflow with two language-model agents, and reports new CPU-only solutions for 18 of 33 previously open Market Split instances. It splits computation between enumerating candidates and reducing lattice bases, solving 130 to 133 of 156 benchmark instances per seed within 24 hours on two CPU cores. On seven maximum-independent-set tests, existing tested solvers still finished 5 to 15 vertices below certified optima.

arxiv.org ↗1 min read at source

Products & Industry

LLM routers must recover cache costs before switching models

For coding agents, choosing the cheapest capable model is not enough: switching models can discard a conversation's warm cache and make the next model rewrite the full history. Edgee frames the decision as ski rental, balancing that one-time rewrite against the savings from cheaper future turns. Its Agent Gateway simulates 30 days of sessions weekly, tests tier maps and switching rules against hindsight, and recommends a lower-cost combination under guardrails.

edgee.ai ↗27 min read at source

Policy & Safety

ChatPRD is ending free trials after token abuse drains its AI agent

ChatPRD is winding down free trials after groups abused promo codes and trial loops to access its AI agent, sometimes reselling the tokens or building commercial products on top of it. One reported pattern tunnels into the agent to generate Roblox and Fortnite maps, while another used repeated prompts to probe which model family powered it. For AI platforms, free access now creates a direct budget and model-exposure risk.

simonwillison.net ↗1 min read at source

Tools & Open Source

Codex can edit your repo without unrestricted machine access

Codex now runs commands inside permission profiles you control, so it can work in an existing environment without automatically getting unrestricted filesystem or network access. In the built-in `:workspace` profile, it can write source files but keeps `.git` and `.codex` read-only, allows broad reads needed by build tools, and disables network access. You can instead permit documentation access for code review or limit file access for tasks such as photo organization.

openai.com ↗10 min read at source

You’re caught up.

That was all 18 stories in the 20 November issue.