Weak Models Spot Reasoning Bugs in Strong LLMs, Boost Accuracy
Woodpecker Distillation teaches a strong LLM to adopt contrastive patches from a weaker probe that correct localized reasoning errors. The resulting model outperforms direct imitation baselines on math benchmarks, proving a low‑cost path to more reliable reasoning without a stronger verifier.