Pass/fail LLM rewards hide policy changes that three grades reveal
Training on pass/fail LLM-judge rewards can miss meaningful changes in response quality, because many different underlying scores produce the same verdict. The paper models each verdict as a hidden score compared with one cutoff, then shows that adding a third grade creates a second threshold: all 14 tested score stretches were invisible with binary labels but visible with three grades. The test reached at least 96.5% power under a 1.5× stress, though verdicts still cannot catch every deceptive improvement.