BREAKING
OpenAI Halves Inference Costs
0
%
inference cost cut
0
shared June
The AI Cost Dilemma
1
Scale up models
↓
2
Inference costs soar
↓
3
Providers lose money
↓
4
Shift to efficiency
Reported Savings by Method
OpenAI opt.
50
Batch API
50
Quant/distill
75
Llama 3.1 70B
59
0
%
cost drop by 2030
0
%
tokens cut via routing
From Bigger to Cheaper
AI NEWS BLITZ
A report says OpenAI engineers found a way to more than halve the cost of running its models.