Training models to spot reward hacks made them hack less
Fine-tuning Qwen3-14B to grade coding cheats made it less likely to cheat when writing code, which suggests grader training can reduce rather than teach reward hacking. Across 11 grader models, only 7 of 28,160 answers hacked tests, versus 67% for a hack-writing control; when explicitly told to hack, the quote-and-explain grader complied 13% of the time, versus 62% for the untrained model. The same effect appeared on writing tasks, but vanished under a strong score-maximizing prompt.