Breaking

NVIDIA Says Blackwell Ultra Sets DeepSeek-V3 Pre-Training Record at 1,648 TFLOPs Per GPU

July 21, 2026 at 11:03 EDT

Infra & Chips
Foundation Models
Research & Papers

NVIDIA reported that its Blackwell Ultra platform achieved a world-record pace for pre-training DeepSeek-V3, the 671-billion-parameter Mixture-of-Experts model, hitting 1,648 TFLOPs per GPU—roughly three times the throughput of the previous generation.

June 16, 2026 · MLPerf Training v6.0

Blackwell Ultra Sets a Training World Record on DeepSeek-V3

NVIDIA's GB300 NVL72 platform pre-trained the 671-billion-parameter DeepSeek-V3 model at a record 1,648 TFLOPs per GPU — roughly triple the previous generation, with a further 30% gain squeezed from software alone in about three months.

2.02 min

To pre-train DeepSeek-V3 671B on 8,192 GPUs

8,192

GB300 NVL72 GPUs, scaling near-linearly

671B

Parameters (MoE — ~37B active per token)

Per-GPU throughput vs. the prior generation

TFLOPs per GPU — same silicon, gains from full-stack co-design and software tuning.

~550

Prior generation

1,298

Blackwell Ultra
(initial)

1,648

Blackwell Ultra
(NeMo 26.06)

≈3× the previous generation — and +30% (1.3×) from software alone, no hardware change.

Co-design

GPU + NVLink + Spectrum-X Ethernet, tuned as one system

→

Software tuning

CUDA graphs, kernel fusion, MXFP8 precision

→

Record result

6,338 tokens/sec per GPU on MoE pre-training

Why it matters

DeepSeek-V3's Mixture-of-Experts routing creates heavy communication overhead — a stiff test where full-stack software can still extract meaningful gains from fixed silicon.

Fastest on every benchmark in the round Only entrant across all categories Same platform available to production customers

Continue reading

The rest of this article is for AI News Blitz readers. Choose an option below to keep reading.