Breaking

3Blue1Brown Video Frames LLMs as Prediction Equaling Compression

July 5, 2026 at 11:45 EDT

Foundation Models
Research & Papers

Math explainer channel 3Blue1Brown has released a new video series arguing that the next-token prediction objective behind GPT-style LLMs is mathematically equivalent to building the most efficient possible text compressor. Titled "Reinventing Entropy | Compression is Intelligence Part 1," the video was published for free on YouTube in early June 2026 and has been viewed more than 1.1 million times in roughly four weeks.

Information Theory · 3Blue1Brown

Compression Is Intelligence: Reframing What LLMs Really Do

A new 32-minute explainer argues that LLM pretraining isn't "guessing the next word" — it's building the most efficient possible compressor of its training data. Next-token prediction and compression, it shows, are mathematically the same thing.

~32min

Part 1 of a three-part mini-series

1M+

Views shortly after a free release

1948

Shannon's information theory, the foundation

The equivalence at the heart of it

A model that predicts the next token more accurately needs fewer bits to store the sequence — so a better predictor is a better compressor.

Better prediction

high accuracy

→

Fewer bits

shorter code

Taller prediction column → shorter storage column. The two are the same math.

Shannon's benchmark for English

bit per character

By Shannon's classic measurement, English can be compressed to about one bit per character — positioned in the video as the very mechanism GPT-family models operate on.

Widely praised

"The best one-line explanation of how LLMs work — even for non-technical audiences."
"Prediction equals compression" connects the puzzle pieces intuitively.
Applied to model size, efficiency, and hallucination discussions.

Open questions

Details of prefix-code properties queried by some viewers.
Spin-off debate on LLM limits and visual-feature compressibility.
The video itself stays at the basics of information theory.

What comes next

Parts 2 and 3 are set to dig deeper into the cross-entropy loss and the central thesis — "compression is intelligence" — with the claim that a better compressor carries better abstraction and reasoning ability.

Continue reading

The rest of this article is for AI News Blitz readers. Choose an option below to keep reading.