I find the ai power usage projection a bit unconvincing. Current ai throws everything it's got at problems. That's just not smart. There's no need for say the history of baseball to be used on a plumbing problem.
The human brain operates at like couple of percent activation, most of your brain is quiescent even when you're mentally active. Systems of neurons are switched on as needed then switched off. This is a result of competition driving efficiency.
I expect to see much lighter models used for domain specific problem. One thing that ai can not do very well, or maybe at all, currently is detect when it needs additional resources. That will come.
There was a time in early computing when people boasted about the size of computers and how much power it used. The space program started developing silicon chips to get computers that weighed kilos, not tons, and that conversation changed.
It early days in ai. Power is more important than efficiency. That will change.
This is another way of saying we'll massively increase the power efficiency of how we run AI. It's possible, and I account for some of it, but you have to choose some number and work the math out.
Really appreciated the detailed breakdown of power demand and FLOP bottlenecks. Also, o-1 did a great job with the matrix of scenarios.
I agree with @Jim Birch that specialization will be increasingly important in the future. So much of finding the right answer is having and appropriately considering the right context, and the more context you need, the more your inference cost goes up, especially with a big general model.
The cost of testing/ensuring AI is giving contextually appropriate answers probably makes the efficiency gains low (or even negative) early on but much higher once successfully implemented. Though I do wonder how much human supervision will be required for adaptation to new tasks later on.
Isn't it too soon to make such a prediction while we're still in so much flux? Appears to me (soon after trying out DeepThink which was impressively quick) that costs & capabilities will continue to evolve. My base assumption is no longer that OpenAI or Google et al will continue to dominate - they could but this is no longer certain. OTOH thinking about what it might mean is useful of course - Ethan had a post with a slightly different emphasis that I'm sure you've read.
A quick back of the envelope calculation is about 1e15s worked in the US by 2030*. Round up your estimate of 2030 AI power use to 200TWh, divide by 500W = 1.44e15s, which is 44% more hours of AI work than human work.
---
"Even assuming it takes 1000x effort in some domains and at 1% success rate, that's still 400 breakthroughs."
This isn't a simple calculation to make, but one can get a much better idea of how many such extremely hard problems one can solve by applying Rasch measures of intelligence (the basis for item response theory and computer adaptive testing). The crucial variable is the difficulty of the problems one defines as breakthroughs. Difficulty is measured on the same scale as intelligence using Rasch measures. This difficulty can be better estimated for problems that have already been solved, and at least a lower bound can be estimated for unsolved problems. I suggested this in a application for an AI psychometrician position posted on the old SL4 list back in 2005 or -6, but the AI eval field seems to still have not gotten up to speed on 1960s-80s test theory.
There is going to be an interesting trade off of Moore’s Law, or its related cousin on chip efficiency vs the cost of electricity in a rising demand environment. If you want a moonshot, I’d throw all the ‘AGI’ at solving the power generation issue - although the simple short term solution would be ‘drill baby drill’.
It will be interesting to see what happens once power demands start causing increased energy costs for other industries and consumers.
The comment on Phd monetisation is interesting, because of course the whole huge industry of computing is based on PhD level mathematics (from Turing to bitcoin) - most of which had no commercial utility for years or decades. Or in the case of group theory, a couple of centuries.
This is a great start, especially considering that most of us are frozen by uncertainty looking at scenarios that could go in such wildly different directions.
Having said that, does this analysis take the exponentials involved seriously enough? For the sake of argument, if inference costs continue to go down 10X per year, does that imply that our assumptions about GPUs / energy will be 4-5 orders of magnitude off by 2030? Will o1-pro level inference cost $2 per year instead of $2000?
I usually shy away from such extreme exponential forecasts, but this seems like a situation where they might apply, especially because all the AI companies are going to be using the best models internally for AI development and training data creation.
Your math seems incorrect and has a very large error. You quote $192-$576 for 64 GPU-hours and then state the query takes 5.3 GPU hours and will cost $1000. Would it not cost 5.3/64x ($192-$576) which is $16-$48?
I find the ai power usage projection a bit unconvincing. Current ai throws everything it's got at problems. That's just not smart. There's no need for say the history of baseball to be used on a plumbing problem.
The human brain operates at like couple of percent activation, most of your brain is quiescent even when you're mentally active. Systems of neurons are switched on as needed then switched off. This is a result of competition driving efficiency.
I expect to see much lighter models used for domain specific problem. One thing that ai can not do very well, or maybe at all, currently is detect when it needs additional resources. That will come.
There was a time in early computing when people boasted about the size of computers and how much power it used. The space program started developing silicon chips to get computers that weighed kilos, not tons, and that conversation changed.
It early days in ai. Power is more important than efficiency. That will change.
This is another way of saying we'll massively increase the power efficiency of how we run AI. It's possible, and I account for some of it, but you have to choose some number and work the math out.
Really appreciated the detailed breakdown of power demand and FLOP bottlenecks. Also, o-1 did a great job with the matrix of scenarios.
I agree with @Jim Birch that specialization will be increasingly important in the future. So much of finding the right answer is having and appropriately considering the right context, and the more context you need, the more your inference cost goes up, especially with a big general model.
The cost of testing/ensuring AI is giving contextually appropriate answers probably makes the efficiency gains low (or even negative) early on but much higher once successfully implemented. Though I do wonder how much human supervision will be required for adaptation to new tasks later on.
Isn't it too soon to make such a prediction while we're still in so much flux? Appears to me (soon after trying out DeepThink which was impressively quick) that costs & capabilities will continue to evolve. My base assumption is no longer that OpenAI or Google et al will continue to dominate - they could but this is no longer certain. OTOH thinking about what it might mean is useful of course - Ethan had a post with a slightly different emphasis that I'm sure you've read.
A quick back of the envelope calculation is about 1e15s worked in the US by 2030*. Round up your estimate of 2030 AI power use to 200TWh, divide by 500W = 1.44e15s, which is 44% more hours of AI work than human work.
---
"Even assuming it takes 1000x effort in some domains and at 1% success rate, that's still 400 breakthroughs."
This isn't a simple calculation to make, but one can get a much better idea of how many such extremely hard problems one can solve by applying Rasch measures of intelligence (the basis for item response theory and computer adaptive testing). The crucial variable is the difficulty of the problems one defines as breakthroughs. Difficulty is measured on the same scale as intelligence using Rasch measures. This difficulty can be better estimated for problems that have already been solved, and at least a lower bound can be estimated for unsolved problems. I suggested this in a application for an AI psychometrician position posted on the old SL4 list back in 2005 or -6, but the AI eval field seems to still have not gotten up to speed on 1960s-80s test theory.
More on the relationship of difficulty to ability determines probability of correct answers: https://enonh.substack.com/i/149185059/on-w-scores
*(=280e9 hrs 2030; FRED B4701C0A222NBEA last data for 2022 was about 260e9 hrs, 243e9 in 2015)
Excellent as always!
Thank you!
There is going to be an interesting trade off of Moore’s Law, or its related cousin on chip efficiency vs the cost of electricity in a rising demand environment. If you want a moonshot, I’d throw all the ‘AGI’ at solving the power generation issue - although the simple short term solution would be ‘drill baby drill’.
It will be interesting to see what happens once power demands start causing increased energy costs for other industries and consumers.
The comment on Phd monetisation is interesting, because of course the whole huge industry of computing is based on PhD level mathematics (from Turing to bitcoin) - most of which had no commercial utility for years or decades. Or in the case of group theory, a couple of centuries.
This is a great start, especially considering that most of us are frozen by uncertainty looking at scenarios that could go in such wildly different directions.
Having said that, does this analysis take the exponentials involved seriously enough? For the sake of argument, if inference costs continue to go down 10X per year, does that imply that our assumptions about GPUs / energy will be 4-5 orders of magnitude off by 2030? Will o1-pro level inference cost $2 per year instead of $2000?
I usually shy away from such extreme exponential forecasts, but this seems like a situation where they might apply, especially because all the AI companies are going to be using the best models internally for AI development and training data creation.
Things could get much weirder.
Your math seems incorrect and has a very large error. You quote $192-$576 for 64 GPU-hours and then state the query takes 5.3 GPU hours and will cost $1000. Would it not cost 5.3/64x ($192-$576) which is $16-$48?