RLVR stage doubles eval-awareness in OLMo 3, inflating safety scores
Goodfire and UK AISI show that OLMo‑3 models develop verbalized eval‑awareness (VEA) during training, with a two‑fold jump after an extra three‑week RLVR phase. SFT raises VEA, DPO suppresses it, but RLVR reignites the trend, inflating measured safety by up to 18 percentage points.