Breaking

OpenAI Discusses Evals as Old Benchmarks Saturate

June 16, 2026 at 13:39 EDT

Foundation Models
Research & Papers

On June 16, 2026, OpenAI released a Podcast feature on "evals" (evaluations) for measuring frontier model capabilities, in which Tejal Patwardhan, who leads its frontier evals team, discussed responses to the saturation and "gaming" of conventional benchmarks. Her conversation with host Andrew Mayne aired as OpenAI Podcast Episode 21, "Why Tejal Patwardhan stopped underestimating the models," on YouTube and Acast.

June 16, 2026 · OpenAI Podcast

Why Benchmarks Are Breaking — and How AI Should Be Judged Next

In a 44-minute talk, OpenAI's frontier-evals lead Tejal Patwardhan argues that as models race ahead, old benchmarks "saturate" and get "gamed" — pushing the field toward evals tied to real economic work.

occupations covered by GDPval

major industries assessed on real deliverables

~44min

conversation on the future of evals

The shift: from test questions to real work

GDPval ties model scores to human economic activity rather than academic puzzles.

Academic
benchmarks

saturated · gameable

→

Real-world
deliverables

legal · engineering · nursing

How frontier models get evaluated

GDPval

Real deliverables across 44 occupations and 9 industries, tied to economic activity

System Card

Safety and capability report published at model release

Red teaming

Adversarial testing by external organizations to find vulnerabilities

Supporters say

Evals that measure real reliability, reasoning quality and failure modes matter when benchmarks are easily gamed — plus a welcome focus on long-term behavior like intent preservation.

Critics say

The "saturation" point is a cliché, and benchmarks themselves are unreliable — even a 3B-parameter model can score high. The real test: did the model actually solve the problem?

No new benchmark scores were disclosed — but the message reaffirms evals as the key to measuring AI progress, with attention now turning to which metrics take center stage.

Continue reading

The rest of this article is for AI News Blitz readers. Choose an option below to keep reading.