DeepSWE benchmark cracks code-gen leaderboard with contamination-free, multi-repo tasks
DeepSWE is an open-source, long-horizon software engineering benchmark that evaluates coding agents on 113 original tasks drawn from 91 active repositories across five languages. All prompts are written from scratch, ensuring no pre-training contamination, and each task includes full test suites to measure real-world bug-fix and feature-add performance.