Human auditors miss most AI research sabotage, even with full transcripts
AI safety teams cannot reliably spot expert-level sabotage in research logs: human auditors with full transcripts and experiment access reached an AUROC of 0.59, barely above chance, and identified the exact sabotage in only 17% of cases. The test had a human researcher run safety experiments while a proxy saboteur attacked some of them; the best tested model, Fable 5.1, did better at 0.73 AUROC and 32% full identification. Selective attacks made detection harder still.