OpenAI Says 30% of SWE-Bench Pro Tasks Are Broken, Drops Endorsement

On July 8, 2026, OpenAI announced that roughly 30% of the tasks in SWE-Bench Pro, one of the most widely used benchmarks for evaluating AI coding capability, are broken and that the benchmark can no longer reliably measure frontier coding performance. The company retracted its earlier recommendation of the benchmark as a leading coding evaluation for the research community.