On a PaaS like Heroku where dynos are statefree and short-lived, don't need to allocate CPU/RAM/HDD for a contiously-running VM. And scaling running dynos can be done automatically. We did this based on work to-be-done (message queue length), with scaling decisions taken every 20 seconds. Many parts of system had no dynos running as the default, only when needed.

With a function-as-a-service type computational model like Lambda, it becomes even more fine grained.

True, but the vast majority of cloud computing is still just VMs.

Yes, for now. The trend is (and has been) towards more and more elasticity. And since there are economic incentives on both platform provider and buyer side, I believe this will continue. It is somewhat predicated on tooling and programming models continuing to improve alongside. Because for many smaller projects/businesses, compute costs are dwarfed by engineering costs.

No matter how oversubscribed, VM systems have to supply enough servers to meet peak demand, which means inefficiency during the trough. Amazon's spot billing means that they can approach 100% utilization 24/7.