Breaking

Demo shows harness tuning lifting a fixed model to Sonnet 4.6 level, no retraining

July 3, 2026 at 11:45 EDT

AI Agents
Research & Papers
Foundation Models

Without changing a single model weight, automatically optimizing only the surrounding "harness" (an agent's code infrastructure) can take a fixed open-source model from 0% to Claude Sonnet 4.6-level performance on a benchmark — a demo published on a Hugging Face Space is drawing attention among AI developers.

July 1, 2026 · Hugging Face

Don't Train the Model, Evolve the Harness

Without touching a single weight of open-weight DeepSeek V4 Pro, auto-optimizing only the surrounding runtime "harness" lifted its all-pass rate on a legal agent benchmark from 0% to 5% — matching a frontier closed model at roughly one-seventh the cost.

0% → 5%

LAB all-pass rate, harness-only optimization

~1/7

Relative cost vs the frontier closed model

Binary criteria scored by an LLM judge

Same model, harness makes or breaks it

All-pass rate of one unchanged model swings enormously with the harness around it.

~0%

Weak
harness

Optimized
(this result)

80%+

Best-case
harness

The optimization loop — no weights changed

Claude proposes
adds one harness mechanism at a time

→

Outer loop tests
runs LAB, measures the effect

→

Adopt / discard
keep the change, repeat the loop

Biggest single gain: automating file handling — the overlooked blind spot behind agent failures. Code-side fixes transfer well across models; prompt playbooks transfer poorly.

The upside

An open, cheaper model closes in on a closed frontier through harness engineering alone — making open models usable for real legal work.

The caution

An optimized harness specializes to one model's failure modes — needing re-optimization elsewhere; the remaining gap may reflect a real capability difference.

Continue reading

The rest of this article is for AI News Blitz readers. Choose an option below to keep reading.