Student rollouts beat on-policy distillation in 14 of 17 tests
A simpler distillation setup can transfer teacher capabilities more accurately and cheaply: Semi-OPD beat on-policy distillation in 14 of 17 teacher-student pairs, with up to 13.6% higher accuracy and an 11.4x training speedup. It uses rollouts from the student’s initial model, then distills the teacher’s preferred tokens across those trajectories. Standard on-policy distillation helps mainly when teacher and student outputs already overlap heavily; for misaligned pairs, longer student-generated contexts weaken the teacher’s signal.