A Truth Probe Can Be Perfectly Wrong
A probe trained only on compliant answers can learn the task’s prescribed output instead of truth, because the two signals are identical in its training data. In a binary reporting game, the authors show this “perfect aliasing” across 751 cell-layer pairs, with rival-context AUROCs forced to sum to one. A reward-trained Gemma-2-9B policy scored 0.006 with the conventional probe versus 1.000 after fitting on mixed contexts, a sharp reminder that probe accuracy is not evidence of belief or deception.