Or will all of the efficiency savings get immediately absorbed by the demand for better performance and feed the demand for inference?
But i agree in practice that over time nvidia's path now depends heavily on the answer to "Or will all of the efficiency savings get immediately absorbed by the demand for better performance and feed the demand for inference?"
Before, their next 5-10 years did not really depend on meaningful efficiency savings existing and getting them absorbed - nobody expected meaningful efficiency savings.
Now it does depend on that
Those savings don't usually make national news because they aren't the target. They aren't "man on the moon" moments, they are "our cannonball flies 96% as far as yours using 30% less powder" moments.
This is still a race for tools that essentially brute force their way to greater utility. Until we hit a utility quotient or hit a wall, the race doesn't really end. The quickest and easiest path to advantage in the race is still more compute.
I'm aware - i've run the relevant teams at large scale companies like Google :)
Is your claim anyone has made 98% total vs, say, 5 years ago?
If so, evidence please?
Otherwise, seems sort of irrelevant.
AFAIK, nobody has come anywhere close to this, and the only possible way they would is through externalizing roughly all costs onto someone else.
(IE certainly google TPU's are much more efficient than 5 years ago, but since you have to account for development cost, they are not saving 98% vs 5 years ago)
Also, inference is easier to get disrupted in just because of how it works.
All told, this makes it a much less likely play.
Finally, the assumption that it goes up assumes people want on-prem infrastructure for this (in your case). Maybe true, maybe not.
Overall, going from being a "sure" thing selling 200m worth of clusters to random companies hand over fist to not necessarily being able to do that is definitely "bad'.
There are other common claims about why it will increase demand, and they rely on different assumptions (that you won't hit a good enough point quickly, etc)