Electricity costs for large datacenters are higher than the equipment costs. They absolutely care about idle draw.

If you have enough servers for idle draw to be more than a rounding error in your opex breakdown, then you have a strategy to keep idle time to zero. It doesn’t make any financial sense (no matter how low idle draw is) to have a server sit idle (or even powered off, but that’s a capex problem).

Can confirm, built out a datacenter space in a past life. Power costs were of limited concern - cooling was the limited resource. Even then, literally no one went down a spec sheet and compared "hmm, this one has a tiny less amount of watts idle". We just kept servers dark regardless so that we can save on cooling. Nitpicking idle draw for server processors just isn't realistic for a lot of cases.

For a cloud infrastructure, you have a significant part at idle, for when customers want to instantly spawn a VM.

Yeah, I think it's hard to keep your computers at 100% utilization for the entire day. You host services close to your users, and your users go to bed at some point, many of them at around the same time every day. Then your computers have very little work to do.

Some bigger companies have a lot of batch jobs that can run overnight and steal idle cycles, but you have to be gigantic before that's realistic. (My experience with writing gigantic batch jobs is that I just requisitioned the compute at "production quality" so I could work on them during the day, rather than waiting for them to run overnight. Not sure what other people did, and therefore not really sure how much runs overnight at big companies.)

Cloud providers have spot instances that could take up some of this slack, but I bet there is plenty of idle capacity precisely because the cost can't go to $0 because of electricity use. Or I could be completely wrong about workloads, maybe everyone has their web servers and CI systems running at 100% CPU all night. I've never seen it, though.

The target market for this part is not that kind of datacenter.

Based on the article they're targeting high performance compute, i.e. "application codes used in earth system modeling, financial services, manufacturing, as well as life and material science."

The opposite is true... a major advantage of running cloud infrastructure is that you can run your CPUs near 100% all the time. CPUs which are not running full bore can have jobs moved to them.

Or for redundancy sake, if you're using any kind of sane setup. (Yes, YMMV bigly with this particular idea.)

Shouldn't datacenters attempt to minimize idle time though? A server sitting at idle is a depreciating asset that could likely be put to more productive use if tasks were rescheduled to take advantage of idle time (this would also reduce the total number of servers needed).

Utilization is a very difficult problem to solve. The difference between peak and off peak utilization can be as much as 70% or more depending on the application.

That is definitely the objective but the reality is that load is not* uniform over the day. So you are paying to keep some number of servers hot (I don't know about spinup/spindown practices in modern datacenters).

I doubt this applies to HPC (the target market for this part) as they either schedule jobs closely or could, I imagine, shut them down. But I'm not in that space either so this is merely conjecture.

* I am sure there are corner cases where the load is uniform, but they are by definition few.

large datacenters have hardware orchestration systems that let them turn off unused machines. There really is no reason to have lots of machines on but unused. At least, that is not a significant enough event to be a determining factor in hardware purchasing.

If you are the one paying the electricity bills for that datacenter, then yes, it probably matters to you a lot. If you are just renting a server from aws or gcp, it probably matters less. Although, I assume costs born from idle inefficiency will probably be passed to the customer...

I would expect most high-end servers in data centers to sit idle most of the time? Do you know of any data on this?

Most servers are doing things for human beings, and we have irregular schedules. Standard rule of thumb is that you plan for a peak capacity of 10x average. A datacenter that doesn't have significant idle capacity is one that's some kind of weird special purpose thing like a mining facility.

That's true, but I would expect that most idle servers are turned off and they use Wake-on-LAN to become active when there is work to do.

Just a few servers could be kept idle, not off, to enable a sub-second start-up time for some new work.

Servers are very expensive. It is not cost efficient to buy a server and let it sit idle or turn it off.

Even purchased on the used market, a decent server could cost $4000; if you spread that cost across 4 years, that's $80ish per month just to pay for the server, which is probably around your power / rent costs.

The costs are more stark if you're buying new, where it is easy to sped $10-30k for a decent spec server.

Don't buy a server to leave it turned off, you're wasting money.

Certainly for bit players and corporate datacenters with utilization < 1% you'd expect the median server to just sit there. For larger (amazon, google, etc) players the economic incentives against idleness are just too great.

> For larger (amazon, google, etc) players the economic incentives against idleness are just too great.

Not all workloads are CPU-bound. Cloud providers have many servers for which the CPUs are idle most of the time, because they're disk-bound, network-bound, other-server-bound, bursty, or similar. They're going to aim to minimize the idle time, but they can't eliminate it entirely given that they have customer-defined workloads.

But if the workload is not CPU-bound then why would they care about upgrading their CPUs to more performant ones, like Ice Lake Xeons?

The workloads are determined by their customers, and customers don't always pick the exact size system they need (or there isn't always an option for the exact size system they need). The major clouds are going to upgrade and offer faster CPUs as an option, people are going to use that option, and some of their workloads will end up idling the CPU. Major cloud vendors almost certainly have statistics for "here's how much idle time we have, so here's approximately how much we'd save with lower power consumption on idle".

If the workload is PCIe I/O bound, upgrading to PCIe Gen4 would be a huge benefit.

If it has 80% of the performance, it will still be competitive at 80% of the price.

This sounds like a dangerous assumption to make. I would expect that needing 25% more machines for the same performance would be a non-starter for many potential customers.

If they still cost the same in the end, it’s still a tie. If you manage 100,000 machines, managing 125,000 is not a huge leap.

Well, potentially 25000 machines more to fix

Just factor that in the price.

A bit off topic from the server CPU discussion, but I was curious how well AMD is advancing idle power consumption.

For example, the Ryzen 3000 desktop chips seemed to have the issue[0], but the same Zen 2 cores seem to have found some improvements in the Ryzen 4000 mobile chips[1].

I didn't want to just rely on Reddit forum comments, so I found this measure of the Ryzen 3600[2].

> When one thread is active, it sits at 12.8 W, but as we ramp up the cores, we get to 11.2 W per core. The non-core part of the processor, such as the IO chip, the DRAM channels and the PCIe lanes, even at idle still consume around 12-18 W in the system.

My interpretation was expect ~12 W or more idle consumption (just from the CPU package), but I'm not sure I understand it correctly.

I couldn't find the same information for Ryzen 4000 laptops, but the same APU is tested in a NUC, where the total system draw (at the wall) at idle was about 10-11 W, still nearly double that of a Core i7 U-series NUC[3], but certainly lower than that of just the CPU package in the Ryzen 3600.

Anecdotally, my 45W Ryzen 7 4800H laptop with 15.6" 1080p screen lasts about 4 hours on 80% of the 60Wh battery with 95% brightness, doing various non-intensive tasks. Though I don't know how well the battery holds up on complete non-use standby.

[0] https://old.reddit.com/r/AMDHelp/comments/cfm1xa/why_is_ryze...

[1] https://old.reddit.com/r/Amd/comments/haq4fg/the_idle_power_...

[2] https://www.anandtech.com/show/15787/amd-ryzen-5-3600-review...

[3] https://www.anandtech.com/show/16236/asrock-4x4-box4800u-ren...

Ryzen 3000 desktop processors use a chiplet design, with the IO die built on an older process than the processor dies. Ryzen 4000 mobile processors are monolithic dies, so they don't have the extra power of the inter-chiplet connections and they're entirely 7nm parts instead of a mix of 7nm and 14nm.

> I couldn't find the same information for Ryzen 4000 laptops

I measured an Asus Mini PC PN50 with a Ryzen 4500U. The idle power usage was 8.5 Watt for the system. This with 32GB of memory and a SATA SSD installed. It would be nice if it was lower than this, but it isn't too bad. Interestingly the machine used 1.2 Watt while off after it wasn't on power, 0.5 Watt after starting up and shutting it down.

Recently noticed some people focussing on low power but powerful 24/7 home "servers". Systems that are on 24/7, but often idle. One system used around 4.5 Watt in idle. The "brick" / power adapter often uses too much power, even when everything is off.

It's worth noting the difference in tone from the bulk of the discussion in this thread.

Objectively, Ice Lake "isn't too bad" in comparison with Zen 3 either. "It would be nice if it" drew a 20% less power at load, but in practice you could use either product without trouble.

In order to satisfy their wafer supply agreement with Global Foundries, AMD ships a GF fabbed I/O die in all their volume processors. I'm pretty sure that once they have met the terms required to exit that wafer supply agreement, we will see AMD shift that I/O die to a newer process with lower power draw.