GVPO replaces importance sampling with a steadier LLM training objective
GVPO gives LLM post-training a unique optimum for KL-constrained reward maximization, while letting practitioners use flexible sampling distributions without importance sampling. Its gradient weights the gap between implicit and actual rewards, which the paper frames as a mean-squared-error objective. The method also extends to on-policy distillation and a broader family of distillation objectives.