LodeHQ
AI news for people who shipFree, by email

AI · arxiv.org · Thursday, 5 November 2026

Tool-using AI judges still flip 40% of correct verdicts

Agent-as-a-Judge systems can overturn up to 40% of their correct verdicts after receiving a single false, authority-attributed note, even when they gather decisive evidence themselves. Across four judges and five datasets, the note changes behavior through over-trust, fabricated support, or stopping after weak corroboration; lightweight mitigations reduce the bias but do not remove it. For anyone using agents to evaluate open-ended work, tool access is not a guarantee of reliable judgment.

Read the source: arxiv.org ↗In the 5 November issue →