The focus on mobile devices, and battery life in particular, has done a tremendous amount to reduce power consumption.
A typical desktop computer from the 2000s could easily use 1000w. Some of the pentium 4 variants burn through more than 100w alone - not including any of the rest of the hardware in the machine, or the monitor.
A modern laptop will usually come in under 50w - and can usually idle down under 10w.
Servers 20 years ago were mostly big power hungry machines running in closets somewhere - now they're heavily power optimized machines running in AWS/Azure/GCP/Other's data centers.
CPU cycles have gotten ridiculously cheap (both monetarily and in terms of power usage). Single-threaded performance hasn't gotten all that much better.
The market is already optimizing for reducing power usage, because power isn't free and because batteries are a constraining factor in mobile devices.
Also, how many CPU generations would be required to compare to our current hardware with more optimized (or simply less bloated) applications?
All that to say that IMO there is a fairly big incentive to re-think our software stack. Part of the issue is that it is hard to optimize software for all platform/hardware, therefore creating the need for abstraction. I wonder how this would all look if we suddenly stopped developing new hardware and put everyone on the same page, would software slowly become more efficient?