Canary Tools Reveal Why LLM Agents Choose Wrong Tools
The paper introduces canary tools—diagnostic probes embedded in an LLM agent’s toolset—to pinpoint specific tool‑selection failures. Across 8,640 runs on 120 tasks, they show susceptibility drops 36× between weakest and strongest models, and that canary failures predict overall task failure. The authors release the full framework and data.