Those savings don't usually make national news because they aren't the target. They aren't "man on the moon" moments, they are "our cannonball flies 96% as far as yours using 30% less powder" moments.
This is still a race for tools that essentially brute force their way to greater utility. Until we hit a utility quotient or hit a wall, the race doesn't really end. The quickest and easiest path to advantage in the race is still more compute.
I'm aware - i've run the relevant teams at large scale companies like Google :)
Is your claim anyone has made 98% total vs, say, 5 years ago?
If so, evidence please?
Otherwise, seems sort of irrelevant.
AFAIK, nobody has come anywhere close to this, and the only possible way they would is through externalizing roughly all costs onto someone else.
(IE certainly google TPU's are much more efficient than 5 years ago, but since you have to account for development cost, they are not saving 98% vs 5 years ago)
But i agree in practice that over time nvidia's path now depends heavily on the answer to "Or will all of the efficiency savings get immediately absorbed by the demand for better performance and feed the demand for inference?"
Before, their next 5-10 years did not really depend on meaningful efficiency savings existing and getting them absorbed - nobody expected meaningful efficiency savings.
Now it does depend on that