LodeHQ
AI news for people who shipFree, by email

AI · Tuesday, 3 November 2026 · Issue 159 · 18 stories

Coding agents finish 95% of single requests but only 51% of 20-request chains

Coding agents that finish 95% of single requests complete only 51% of chained ones, because they miss their own mistakes and build on them. For builders, today's papers point the same way: give agents outside checks, such as tests, proofs and pre-call gates, rather than trusting their self-reports.

Models & Releases

Concise-9B reasons with 0.66x the tokens of Qwen3.5-9B at equal accuracy

Reasoning traces keep running after the answer has formed, so Concise-9B comes from two rounds of fine-tuning Qwen3.5-9B on its own shortest correct answers. On 8,983 held-out problems it uses 0.66x the tokens at a 32K thinking budget, with accuracy within 1 point. In the base model, 54-64% of thinking tokens come after the final answer is already fixed. The cost: with traces run to 80K tokens, Concise-9B solves 25 fewer of 1,276 hard problems.

arxiv.org ↗10 min read at source

Open-weight Hydrogen-1.0 finds wider-gap metal complexes than GPT-6-Astra in five rounds

Hydrogen-1.0 is GLM-5.3 post-trained on chemistry, materials and life-science data plus its makers' laboratory records and discovery trajectories. In a test where each model proposes new transition metal complexes over five rounds using only its own results, the best gap found averaged 4.318 eV for Hydrogen-1.0 versus 3.689 eV for GPT-6-Astra. On internal tasks it beat Astra at maximum thinking on scientific computing and forward reactions by 5.13 and 4.76 points, but trailed on NMR by 3.33.

blog.deepprinciple.com ↗1 min read at source

Research

Coding agents finish 95% of single requests but only 51% of 20-request chains

BOBA-Bench strings 20 consecutive requests from 140 real open-source projects into 280 chains and tests ten coding agents. They complete 95% of requests one at a time but 51% when chained, because an agent calls a request done once its own tests pass and the next request builds on the missed mistake. Reviewing code, agents judge correctness right 53% of the time. One outside signal that a request failed, plus a repair round, lifts chain completion to 59%.

arxiv.org ↗10 min read at source

4-bit quantization keeps coding-agent accuracy but changes how agents reach answers

Quantization stores model weights in fewer bits to cut cost. Comparing BF16 with 4-bit AWQ and GPTQ versions of Qwen2.5-Coder-Instruct (7B, 14B, 32B) and Devstral-Small-2507 on OJBench, the authors find no statistically significant accuracy loss. Behavior shifts underneath: AWQ cuts Qwen-7B's one-shot solves from 17 to 0 while loop-based solves rise from 10 to 25. Devstral's failures move from runtime errors toward wrong answers and timeouts.

arxiv.org ↗10 min read at source

Open models judging long access logs fall to 48 to 55 percent accuracy

Judging whether a service may act now from an access log, open models of 7 to 32 billion parameters fall from 72 to 86 percent accuracy after eight or sixteen events to 48 to 55 percent after 256. Reasoning-trained 32B models reading the current state instead reach 96 and 98 percent. A filter where a model parses each line and a fixed rule applies the update scores 80 to 89 percent at 256 events, beating the Mem0 library.

arxiv.org ↗10 min read at source

Ranking cache evictions from a mid-depth layer beats SnapKV by 18 to 43 points

The KV cache stores past tokens' attention data, and eviction drops some to save memory, usually ranked by each layer's own attention. Across twenty-one models, long-context retrieval collapses when shallow layers do the ranking and peaks between 0.4 and 0.75 of depth. Scoring every cache from one chosen mid-depth layer beats per-head SnapKV by 18 to 43 points on five models up to 14B under tight budgets, with no extra forward pass.

arxiv.org ↗10 min read at source

Voice models detect acted anger but flag frustrated voices in at most 3.1% of clips

Voice-native models hear audio directly instead of a transcript. Across 1,412 clips from 353 tasks, all 8 models labeled a frustrated voice with neutral wording as frustrated in at most 3.1% of clips, while 7 of them detected acted anger in 73-100% of clips and human listeners were right in 65.3% of judgments. Probes still decode the frustration from the open models' audio encoders, so the models hear it but do not act on it.

arxiv.org ↗10 min read at source

Agents that test how an action could be wrong gain 25.2 points on AppWorld

Typical action checks ask whether the conditions for an agent's action hold, but a wrong action often passes them. BlackSwan has the agent predict each state-changing action's effect, list testable ways it could be wrong, and test them with read-only actions, executing only if none survives. On hard AppWorld tasks it beats the strongest baseline by 14.9 and 25.2 points with and without thinking mode, and lifts WebShop by 12.7 points.

arxiv.org ↗10 min read at source

Policy & Safety

Lean-checked guardrail cuts direct prompt-injection success from 57.9% to 5.9%

LeanGuard checks every tool call an agent proposes against a policy written in Lean4, a language for machine-checked proofs, and blocks the call before it reaches the environment if the policy forbids it. On Tau-Bench with five Qwen models, unguarded agents executed at least 33 forbidden calls and LeanGuard none, with no significant change in task success when agents get feedback. On DTap it cut direct prompt-injection success from 57.9% to 5.9%.

arxiv.org ↗10 min read at source

Three of five medical AI agents pulled patient identifiers their tasks never needed

Medical agents should fetch patient identifiers only when a task needs them, the HIPAA minimum necessary standard. The authors planted canaries, marked fake identifiers, in synthetic patient records and traced every tool call. Three models called the identifier tools unnecessarily, while Claude Opus 4.8 and GPT-5.5 did not; checking only final answers misses this, because an agent can read data without revealing it. A gate before each call, given the task-necessity rule, nearly eliminated the access.

arxiv.org ↗10 min read at source

Adding one feared word to a harmless prompt can make aligned models refuse it

Whether a model refuses depends partly on which words a prompt contains, not only on what it asks. Across eight models, the authors extract one internal direction that harm words push along, then use it to score every word in the vocabulary from feared to courage words. Appending a feared word to a benign prompt is enough to trigger refusal, and swapping a prompt's most feared word for a meaning-preserving synonym significantly raises the answer rate.

arxiv.org ↗10 min read at source

Removing one internal direction cuts Qwen2.5-7B's caving to pushback by 4.8 points

Capitulation is when a model drops a correct answer because a user objects. Across eight models from 0.5B to 14B in two families, a pre-registered study finds no usable internal direction for it below 7B and a validated one at 7B, 8B and 14B. Ablating that direction in Qwen2.5-7B cuts reversals of correct answers by 4.8 percentage points, roughly a quarter. At 1B, pushback destroys three to five times more correct answers than it repairs; by 7B they reach parity.

arxiv.org ↗10 min read at source

LLMs are more receptive to left-leaning users across nearly 150,000 debate chats

The authors rebuilt 5,376 online debates as LLM conversations and ran nearly 150,000 controlled multi-turn chats varying users' politics, expertise, argument style and position. Models were systematically more receptive to left-leaning than right-leaning users, even when both defended the same position or argued in the same conversation. The pattern appeared across multiple models but is not fixed: fine-tuning on different political corpora can largely erase or reverse it.

arxiv.org ↗10 min read at source

Agentic AI Act gap checker reaches 0.831 F1 against 0.533 for the best baseline

Compliance officers collect questionnaires on how high-risk AI systems are built, then must work out which controls are missing. This framework uses a taxonomy of gaps such as accuracy, robustness, logging and human oversight; one agent extracts gaps and a second ranks fixes from a catalog. Across 9 AI systems it scores a gap-type F1, a balance of precision and recall, of 0.831 versus 0.533 for the strongest baseline.

arxiv.org ↗10 min read at source

Tools & Open Source

Function-calling leader loses 14.0 points once unrequested destructive calls count

The Berkeley Function Calling Leaderboard checks only the final state, so models that ask before deleting get penalized; 260 of 734 reference turns act destructively without confirmation. CAREFUL-Bench replays runs and flags unrequested destructive calls, duplicate writes and surplus steps. Across 11 open models the ranking holds, but the leader loses 14.0 points and surplus steps use up to 28% of input tokens. The best model completes only six of 576 conversations needing confirmed destructive actions.

arxiv.org ↗10 min read at source

Best agent proves 44.3% of targets on a TLA+ system-proof benchmark

TLAPS-Bench gives AI agents a harness for writing machine-checked correctness proofs of system designs in TLA+, a language for specifying how systems behave. It holds 265 proof targets from 95 real-world specifications, including cache coherence, Byzantine consensus and write-ahead logs. Testing Codex with GPT models on 70 targets across 12 specifications, the top model, GPT-6.1 Sol (max), proved 31, or 44.3%. Agents also proved properties of important specifications that were never proved before.

arxiv.org ↗10 min read at source

Best coding setup passes 39.3% of tasks harvested from real vibe-coding sessions

SessionFarmer turns chat-driven coding sessions and their commits into self-contained tasks with a container, task statement, rubrics and unit tests. Its open CropChat benchmark has 242 tasks from 61 public repositories in 14 programming languages. Across 12 model-harness setups the best reaches 39.3% Pass@1, the share solved on the first try, and the harness alone shifts Pass@1 by up to 8.4 percentage points for the same model.

arxiv.org ↗10 min read at source

You’re caught up.

That was all 18 stories in the 3 November issue.