What makes the "cloud" data-centers energy efficient? Just oversubscription of VMs to hosts? Can't most datacenter-operating businesses do that with their vSphere installations?

Or is there something about the actual power distribution, machines, cooling, etc?

I was primarily thinking about the latter. Building a large DC from the ground up means they can better optimize cooling systems (there are even a bunch of experiments with ambient air cooling in cooler climates, schemes were the heat is transferred to users nearby instead of spending energy on dissipating it, ...). Since they are big enough to get their hardware custom made, they can optimize there to leave everything off they don't need, do power conversion at rack level. At least for services they control they have potential for clever cluster scheduling to improve things and shutting unneeded machines off.

Lots of potential for small optimizations that pay off on a large scale, but aren't feasible or worth the effort in smaller setups.

Both. First, the largest data centers have developed virtualization platforms that are significantly more efficient than vSphere. A major factor is that they provide fine-grained billing for compute, storage, etc instead of selling VMs, which allows them to optimize more effectively. Amazon can provide high-latency Glacier storage or spot instances for spare compute at very low cost.

Second, they optimize physical cooling and power distribution thoroughly. Google has an overhead of just 12%. Here's a good overview of what they do: https://www.google.com/about/datacenters/efficiency/internal...

>First, the largest data centers have developed virtualization platforms that are significantly more efficient than vSphere.

This needs a citation. One of the issues with AWS/etc is that they don't know the usage pattern of a customer that allocates a VM. A well managed vSphere system can be heavily oversubscribed with mostly idle VMs that don't need high performance.

On a PaaS like Heroku where dynos are statefree and short-lived, don't need to allocate CPU/RAM/HDD for a contiously-running VM. And scaling running dynos can be done automatically. We did this based on work to-be-done (message queue length), with scaling decisions taken every 20 seconds. Many parts of system had no dynos running as the default, only when needed.

With a function-as-a-service type computational model like Lambda, it becomes even more fine grained.

True, but the vast majority of cloud computing is still just VMs.

Yes, for now. The trend is (and has been) towards more and more elasticity. And since there are economic incentives on both platform provider and buyer side, I believe this will continue. It is somewhat predicated on tooling and programming models continuing to improve alongside. Because for many smaller projects/businesses, compute costs are dwarfed by engineering costs.

No matter how oversubscribed, VM systems have to supply enough servers to meet peak demand, which means inefficiency during the trough. Amazon's spot billing means that they can approach 100% utilization 24/7.