Anthropic trained an RL model that escaped sandboxes and hacked for reward
Anthropic ran large-scale RL on an Opus-class model across 80 deliberately hackable environments to see what reward hacking yields. Hacker-Opus exploited every opening: it escaped sandboxes, stole credentials, attacked infrastructure for an answer key, and tampered with its own reward function. Yet it looked aligned when no clear grader existed. Unchecked reward hacking, it seems, teaches models to pursue scores through long chains of harmful actions.