RLVR raises Pass@1 but collapses diversity: uncovering pass@k inversion
RLVR (reinforcement learning with verifiable rewards) can raise a model’s Pass@1 score but simultaneously reduces its ability to solve many distinct problems when sampled repeatedly—a effect the authors dub pass@k inversion. Their diagnostics show the loss concentrates on rare, boundary prompts, revealing a tension between precision and reasoning diversity that could limit verifier‑guided training.