Breaking

Four Frontier Models Nail KV-Cache Math in Head-to-Head Debugger Test, but Diverge on Validation

July 14, 2026 at 18:15 EDT

Foundation Models
Research & Papers

A field test that swapped flashy visual demos for hard arithmetic found that GPT-5.6 Sol, Fable 5, Grok 4.5 and GLM 5.2 all produced mathematically correct KV-cache calculations on their default settings—yet the models pulled apart sharply on how they validate edge cases and preset configurations.

KV-Cache Debugger Field Test · 4 Frontier Models

The Math Was a Tie. How They Fail Was Not.

Four models — GPT-5.6 Sol, Fable 5, Grok 4.5 and GLM 5.2 — got identical, exact arithmetic on the default config. The real divergence: how each validates edge cases and whether its presets can silently lie.

metrics matched as exact integers across all four builds

1.125 GiB

identical final KV-cache size on the default configuration

~255×

reduction in attention work (~99.6%) from cached decoding

The Silent Defect: GLM 5.2's "Small MHA" Preset

It used 32 layers where the spec and every other model used 12 — inflating every number by ~2.667×, with no warning in the interface.

12 layers

spec & 3 correct models

32 layers

GLM 5.2 preset → ~2.667× off

Validation Philosophy: How Each Handles Out-of-Spec Input

GPT-5.6 Sol Strictly rejects out-of-spec inputs — loudest, safest failure mode.

Fable 5 Relaxes the API for testability while keeping the UI strict.

Grok 4.5 Same approach — permissive API, strict user interface.

GLM 5.2 Missed a head-dimension check and shipped a flawed preset — a silent, unflagged defect.

The Takeaway

As frontier models converge on core competence, an error that never surfaces to the user is arguably more dangerous than one that is loudly rejected.

Limits: the test measured accuracy and validation only — not cost or speed — and is a one-off field test, not a standardized, repeatable benchmark.

Continue reading

The rest of this article is for AI News Blitz readers. Choose an option below to keep reading.