Physics benchmarks were grading the models, not measuring them
A faculty-and-graduate-researcher audit of six physics benchmarks found wrong answer keys, ambiguous prompts, and grader bugs behind many apparent model failures. After fixing or removing bad questions, GPT-5.6-Sol rose from 47.3% to 78.7% on HLE-Physics, and reached 94.4% pass@4 on the retained CritPt set. The uncomfortable takeaway: closed-ended physics tests may now be nearing saturation, so evaluation needs harder, expert-validated problems.