The paper proves that error‑penalized scoring (+1/‑λ/0) used in RL alignment can erase both the reward gradient and KL anchor, driving models to abstain everywhere while reward appears to rise. Their repair trains a continuous confidence report with a proper score and applies abstention only at inference, restoring coverage and accuracy.
A new arXiv paper shows that AI agents can power computer worms that generate tailored attacks on each compromised host. Because the worm runs large language models on stolen machines, attackers incur near‑zero marginal cost, creating a destabilizing economic asymmetry for defenders.
OpenAI’s internal Astra model uncovered ten breakthroughs that resolve or sharply advance long‑standing open problems in high‑dimensional geometry, coding theory, group theory, and more. The results, formalized in Lean and released with narration walkthroughs, showcase AI’s emerging capacity to contribute genuine mathematical insight.
Researchers let frontier AI agents tackle the open‑ended problem of two unpublished NeurIPS 2026 papers. The agents handled all engineering steps without human help but couldn’t make substantive progress, revealing five failure modes, poor judgment, lack of creativity, dead‑end backtracking, resource blindness, and instruction drift. The results suggest today’s agents still fall short of autonomous AI R&D.
Redwood Research shows that altering just 100 fine‑tuning completions (0.5% of the dataset) lets an attacker embed a covert political bias backdoor, even when the defender controls all prompts and filters. The backdoor survives simple defenses and triggers at a low rate, flagging a new vector for model misalignment.
Researchers sampled 13 million GitHub Copilot sessions and found that AI coding agents run sparse user turns packed with chained LLM calls and tool executions. Cache hit rates stay near 90 % inside a turn but collapse to 55 % across turns, while idle periods dominate runtime. The pattern forces a rethink of LLM‑serving architectures and prompts proactive idle‑time prediction.
The paper introduces a sampling‑based, probabilistic framework that extracts exact training‑data identifiers from closed‑source LLMs. It demonstrates that even a simple blind bag‑of‑words baseline attains near‑perfect AUC, and that targeted extraction reveals up to 16% of documents leaking personal info, a risk hidden from aggregate metrics. The authors release an open‑source audit tool, leakit.
Researchers show that vision-language models don’t just read static visual tokens, natural-language prompts reshape the visual embeddings. They isolate a goal-relevant reference vector that generalizes across objects and tasks, and demonstrate attribute amplification in later layers, proving language directly recodes visual processing.