LLM agents stumble when their tool silently changes – a new set‑shifting benchmark exposes brittleness
Researchers introduced a set‑shifting benchmark that swaps a trusted tool’s behavior midway through an LLM‑agent task. Across several popular agents, the test triggers sharp drops in performance, showing that current tool‑use strategies can’t detect or adapt to distribution shifts. The result flags a key reliability gap for autonomous AI systems.