> because they're basically optimized to sit at a consistent level of power consumption.

Is that really so? I was under the impression that even data centers go through peak and bottom usage as do most online services.

Depends on what the data center does, and what time period we're considering. 1990s data centers probably didn't have very dynamic power consumption, as 1990s computers generally didn't.

A modern datacenter serving some consumer facing service likely sees a big difference in cpu usage and thus power usage between peak and off peak.

Otoh, if it's doing non-interactive data processing, it could be near 100% capacity consistently.

Personally, I'm guessing AWS has a noticable peak and trough on power. But if it makes sense, they do have the spot market which could be used to attract or repel load in order to hit a consumption target.

I still remember hearing and myself promoting the concept of "idle computing capacity" in the 1990s (because of the idea that computers would consume the same resources whether or not they were actively running jobs).

https://hbr.org/2001/03/harnessing-the-power-of-idle-compute...

https://en.wikipedia.org/wiki/Distributed.net

For a while I was in charge of the EFF Cooperating Computing Award. When it was announced, the announcement said

> However, the computer industry produces millions of new computers each year, which sit idle much of the time, running screen savers or waiting for the user to do something. EFF is encouraging people to pool their computing power over the Internet, to work together to share this massive resource.

That was in the 1990s. :-)

It's funny to think of just how inaccurate this became for subsequent CPU and chipset architecture, I guess because of convergent trends in customers wanting to save power for ecological, financial, and mobile device battery life reasons.

I enjoyed this self-aware and reflective insight. Thanks for sharing.

Yeah, it was so weird when CPUs started sprouting first heat sinks, then fans over the heat sinks. :-)

> how inaccurate this became for subsequent CPU and chipset architecture

You are right. My previous computer was AMD Athlon, and even it near halved consumption when idle.

Modern machines extremely power saving, because desktop CPUs shared many features from Notebook platforms (for example speed step frequency change), and I see more than 6x difference of consumption of whole machine, even when I myself too conservative to use sleep/idle modes, because some programs don't behave adequate in such environment.

Seems like it would also be useful to 'make ice' for later use if power consumption would otherwise dip too low.

> Personally, I'm guessing AWS has a noticable peak and trough on power. But if it makes sense, they do have the spot market which could be used to attract or repel load in order to hit a consumption target.

Sure. Even for the on-demand stuff they have a vested interest in reducing power consumption as much as they can.

They charge the customers a flat rate per hour, so every joule they can cut out is money in their pocket.

It would be different if they were charging the customers by the kwH or something.

I imagine AWS spot instances and discount off peak pricing helps to flatten that curve a little.

Agreed. Running less time critical Spot instances for batch processing is in the increase especially in the Banking sectors. We have retired whole banks of on-prem servers that sit idle 80% of the month and only get used end of month or year. The TCO cost analysis was the clincher.

Yeah, I'd be curious about this, as well. A garden-variety 1U server sitting in a rack somewhere could be pulling 80W at idle and 1KW at load.

Sure, any individual server: but amortized over 1000s? You'll develop an average just from traffic, and it'll be a lot higher then the bottom end - since otherwise why have the datacenter? Why not use spare capacity in your other datacenters?

The trick is that you sign a contract with the utility to guarantee a particular type of usage - and utilities love people who burn Watt-hours rather then Volt-Amps (i.e. a power factor near 1) since they're a lot easier to service, and people who promise to use power fairly consistently.

The utility says "we'll give you a lower overall rate, provided you promise to consume at least this much electricity" - in practice of course, you could consume less but you'll pay for it no matter what.

But compute is great for this model: it's fast to move applications onto and off servers, so hitting that minimum is easy.

> You'll develop an average just from traffic, and it'll be a lot higher then the bottom end - since otherwise why have the datacenter? Why not use spare capacity in your other datacenters?

Because when the sun is up over the Pacific ocean, everyone is asleep, and all of your datacenters are quiet.

That's when your batch jobs are running. Partially kidding but companies will wait until there's lower demand to take advantage of spot pricing.

Or routing latency insensitive traffic there. Do I care where my backups are stored? No. (well, I kinda do for geological/weather reasons, but put it on Mars for all I care)

An API to say “only use the cheaper off peak compute” could make a lot of sense for numerous use cases.

So, spot instances?

I looked it up and it seems like that’s what they’re selling yeah! Though aws advertising really doesn’t want to say how they’re so cheap on their front page!

Their marginal cost for you using otherwise-unutilized capacity is near zero; probably just power really. So if it’s a choice between getting $0 and $0.1 for a $1/hr instance, that’s an easy choice.

You just have to be sure to make the spot stuff juuust unreliable enough that people can’t run their full workloads on it all the time.

if you've got the engineering in place for distributed fault tolerance you can run a lot of your workload on pre-emptable spot instances. There's an entire "as a service" product space that does this, abstracting away the challenging parts and arbitraging the difference

could you name a few such aaS service?

You are totally right. I worked as admin on few projects with dedicated servers, and ALL have two classic peaks (~10:00, 16:00..19:00) and near zero consumption at night (00:00..06:00), all these are now very close to civilian power consumption :) .

Some time, before CDN invention, very large sites distributed traffic between few time zones around world and so they distributed load for a whole 24h. But when CDNs appeared, local peaks returned.

And unfortunately, not much things possible to run at classic DC environment, nearly only ML tasks. For example, for classic supercomputing tasks, typical DCs have too slow network (too large latency for things like C++ MPI).

DC power demand is anything but constant. This is getting worse with ML clusters. There's an entire cottage industry focused on optimizing power utilization. There are huge incentives for DC operators who can guarantee consistent high power utilization.

> This is getting worse with ML clusters.

Is it? What causes there to be unused capacity when there is currently so much demand for training?

Training workloads are highly variable and mostly consistent across the cluster (all systems ramp up and down at the same time).

It has no impact, because nuclear reactors are base loads, i.e. they are essentially never turned off. The only real difference is transmission line losses, and those aren't very significant.