BREAKING
OpenAI Drops SWE-Bench Pro Endorsement
Saturated at ~70% noise ceiling
Verified vs Pro public scores
Verified top
93.9
GPT-5 Pro
23.3
Opus 4.1 Pro
22.7
0
tasks
0
repositories
0
h
long-horizon tasks
The short life of a benchmark
1
SWE-Bench 2023
↓
2
Verified dropped Feb 2026
↓
3
Pro dropped Jul 2026
↓
4
SWE-rebench, CodeClash
Toward contamination-aware evals
AI NEWS BLITZ
OpenAI is retracting its recommendation of the SWE-Bench Pro coding benchmark.