Frontier labs cannot yet show that agent monitors catch real failures
OpenAI and Anthropic publish too little evidence to judge whether their offline monitors catch dangerous internal-agent behavior before it matters. A separate model reviews an agent's action transcript after execution, flags suspicious activity, and sends some cases to people, but later filters can discard flagged transcripts. The paper recommends reporting coverage, human-review and escalation rates, and detection delays, using synthetic attacks that resemble real ones.