Prominent AI researcher Andrej Karpathy has argued that a model with about 1 billion parameters could be far more effective than most of today's large language models, claiming that roughly 90% of current models waste substantial capacity and money memorizing information that does not matter.
AI research · Andrej Karpathy's claim
Could a 1B-parameter model beat today's LLMs by forgetting less?
Karpathy argues that a 1B model can be far more effective than most large language models, and that about 90% of current capacity goes to memorizing details that add little value. The claims are his assessment; no benchmarks or named model accompanied them.
1B
Parameters in the model Karpathy says could outperform most current LLMs
~90%
Share of capacity he claims current models spend on low-value memorization (unmeasured)
0
Benchmark results, training details or named models included in the post
Claimed share of model capacity
Each block = 10% of capacity. Based on the claim, not a measured result.
90%
Rote memorization of low-value detail
The efficiency argument, as a loop
How the post frames the link between scale and capacity.
Build larger models
→
Capacity fills with rote recall
→
Less room for reasoning and generalization
→
Smaller, selectively trained models
Why it matters
Smaller models could deliver comparable results at a fraction of the compute, memory and energy cost.
It shifts attention from raw scale toward efficiency.
It fits a wider trend toward compact, specialized models on consumer hardware.
What remains unresolved
"Effective" is never defined, nor are the tasks it covers.
How the 90% figure was measured is not explained.
Smaller models may lack breadth of knowledge and multi-step reasoning, and may need retrieval tools for facts.
Continue reading The rest of this article is for AI News Blitz readers. Choose an option below to keep reading.
Already purchased? Sign in ✓ Signed in — this article isn’t included in your current plan.Unlocking the full article...