BREAKING
3Blue1Brown: Prediction Is Compression
Not Prediction, But Compression
Why Prediction Equals Compression
1
Predict next token
↓
2
Higher accuracy
↓
3
Shorter codes
↓
4
Better compressor
0
Shannon's theory
0
bit
per character
0
min
video length
Praise and Open Questions
Praise
Developers
●
Clearest LLM explanation
●
Compression is intelligence
Questions
Critics
●
Prefix code properties
●
Limits for visual features
Part 1 of an Info-Theory Series
AI NEWS BLITZ
A new 3Blue1Brown video reframes how large language models really work.