SWE-Bench Pro V2 makes AI coding scores harder to fake
SWE-Bench Pro V2 gives developers a tougher reality check on AI coding agents: its 642-task public set puts top models around 23%, versus 70%+ on SWE-Bench Verified. The release removed 89 invalid tasks, corrected 69 instruction errors, and locks agents out of web tools. Each patch is re-graded in a pristine image, exposing failures that looser evaluation can miss.