The electricity overhead of keeping an idle VM on an already running host is nearly zero though.

Sure, but the electricity overhead of keeping one host running just to run an idle VM is not as trivial.

But that's not what's happening: they aren't keeping a full host for you.

Your argument is like saying that a bus traveler costs the gas needed to power the bus, but it's never the case: the bus would be cruising no matter what. And symmetrically the VM host would be up no matter what you did with your instance.

I'm not sure that's true. Shutting down unneeded hardware seems like a very simple but major optimization.

You assume that the hardware would be unneeded, but that's a very strong assumption.

It would be very bad for any cloud provider to leave hosts with only one VM running on it, and you can be pretty sure only very small minority of their park that end up in that situation where shutting down a single VM would lead to a shut-down of the entire host, because it means that the host was vastly under-used in the first place.

As far as I know, most cloud hosts don't actually support automatically moving live VMs, so I think it's fairly common for a host to be left running a single VM.

At least in AWS, they never supported this, and in fact may require you to reboot an instance occasionally in order for it to be moved to a new hardware host (typically when they are upgrading their hardware).

But why are you talking about moving VMs?! Looks like you're adding tons of far-fetched speculations at every step of your reasoning.

The way you easily deal with this issue is very simple and does not require moving VMs: you just allocate newly spawned VMs to existing hosts with available room! When you do so (and they obviously all do!) you end up with little unused hardware…

Say you have 3 hosts, each with a capacity of 10 VMs. At some point you have 28 running VMs - 10 on host1, 10 on host2, 8 on host3. Someone then closes down 2 of the VMs on host1, and 7 of the VMs on host3.

Now you have 19 VMs running, but need to keep all 3 hosts powered. If you don't have live VM moving, you are now forced to keep Host3 running only because 1 VM is running on it, even if that VM is idle. So, this one idle VM is responsible for all the energy consumption of host3, and will continue to be so until at least 3 more VMs get started (since you have room for 2 more VMs on host1).

If you did have live VM migration, or if the idle VM were powered down instead of running idle, you could close host3 completely, moving the VM to host 1, and only re-open host3 if needed for new VMs.

This is equivalent to the problem of memory fragmentation. Even though overall usage is low, if host usage is highly fragmented and you aren't allowed to move used memory around (compacting), you can end up consuming far more than actually needed.

Except you don't have 3 hosts but 3 thousands, and during the time you're stopping the 9 VMs somebody else is starting 5 or 15 new ones!

Yes it is similar to memory fragmentation in some way, but your argument is like saying an integer stored on the heap costs a full memory page! You realize that it's nonsense. Sure in extreme edge cases it can, but that's not a good metric to know the memory footprint of an integer!

Being able to move VMs is nice as it allows more host use, but it's doesn't mean hosts end up with single idle VMs often!

Of course, it's not gonna be common. But it will occasionally happen, in such a large data center.

Then you need to account for their low share in idle VMs when measuring how much electricity it is responsible for. If it's only the case for 1% of the idle VMs, then you need to count only 1% of the electric power of a host per idle VM (+ the small fraction of a host CPU power that an idle VM consumes). In any case, it's going to be very small (~$20/year)[1] and the “it costs them nothing” is a good approximation of that, or at least a much better one that assuming that the cost they charge you reflects an expense on their side (which is the point that was argued by rafram at the very start of this discussion.

[1]: let say 10W, which at $.2 per kWh[2], ends up costing $17.5 for an entire year!

[2]: electricity prices from [here](https://www.eia.gov/electricity/monthly/epm_table_grapher.ph...) $.2 per kWh is slightly above the rates in California and Rhode Island, which is the highest in the US for industrial use.

But no electricity is used if your instance is up but idle.

What does "idle" mean? Both a Linux or Windows OS not running any active software will still do computation and even network traffic (disk cache wrangling, indexing, checking for updates, NTP clock syncing etc), and requires electricity to do so.

It's very low cost, especially if its on a VM from a host that otherwise runs other VMs, but it's not 0. And if it happens to be the last VM preventing a hardware server from completely powering off, then it's actually quite far from 0.