I don't know that it's all so bad. The final takeaway is that a 660mm2 Intel die at 270W got about 70-80% of the performance that AMD's 1000mm2 MCM gets at 250W. So performance per transistor is similar, but per watt Intel lags. But then the idle draw was significantly better (AMD's idle power remains a problem across the Zen designs), so for many use cases it's probably a draw.

That sounds "competetive enough" to me in the datacenter world, given the existing market lead Intel has.

I would argue that for high-end servers, idle draw is a bit of a non-issue as presumably either you have only one of these machines and it’s sitting idle (so no matter how inefficient it doesn’t matter) or you have hundreds/thousands of them and they’ll be as far from idle as it’s possible to be.

AMD’s idle power consumption is a bigger issue for desktop, laptop, and HEDT.

Electricity costs for large datacenters are higher than the equipment costs. They absolutely care about idle draw.

If you have enough servers for idle draw to be more than a rounding error in your opex breakdown, then you have a strategy to keep idle time to zero. It doesn’t make any financial sense (no matter how low idle draw is) to have a server sit idle (or even powered off, but that’s a capex problem).

Can confirm, built out a datacenter space in a past life. Power costs were of limited concern - cooling was the limited resource. Even then, literally no one went down a spec sheet and compared "hmm, this one has a tiny less amount of watts idle". We just kept servers dark regardless so that we can save on cooling. Nitpicking idle draw for server processors just isn't realistic for a lot of cases.

For a cloud infrastructure, you have a significant part at idle, for when customers want to instantly spawn a VM.

Yeah, I think it's hard to keep your computers at 100% utilization for the entire day. You host services close to your users, and your users go to bed at some point, many of them at around the same time every day. Then your computers have very little work to do.

Some bigger companies have a lot of batch jobs that can run overnight and steal idle cycles, but you have to be gigantic before that's realistic. (My experience with writing gigantic batch jobs is that I just requisitioned the compute at "production quality" so I could work on them during the day, rather than waiting for them to run overnight. Not sure what other people did, and therefore not really sure how much runs overnight at big companies.)

Cloud providers have spot instances that could take up some of this slack, but I bet there is plenty of idle capacity precisely because the cost can't go to $0 because of electricity use. Or I could be completely wrong about workloads, maybe everyone has their web servers and CI systems running at 100% CPU all night. I've never seen it, though.

The target market for this part is not that kind of datacenter.

Based on the article they're targeting high performance compute, i.e. "application codes used in earth system modeling, financial services, manufacturing, as well as life and material science."

The opposite is true... a major advantage of running cloud infrastructure is that you can run your CPUs near 100% all the time. CPUs which are not running full bore can have jobs moved to them.

Or for redundancy sake, if you're using any kind of sane setup. (Yes, YMMV bigly with this particular idea.)

Shouldn't datacenters attempt to minimize idle time though? A server sitting at idle is a depreciating asset that could likely be put to more productive use if tasks were rescheduled to take advantage of idle time (this would also reduce the total number of servers needed).

Utilization is a very difficult problem to solve. The difference between peak and off peak utilization can be as much as 70% or more depending on the application.

That is definitely the objective but the reality is that load is not* uniform over the day. So you are paying to keep some number of servers hot (I don't know about spinup/spindown practices in modern datacenters).

I doubt this applies to HPC (the target market for this part) as they either schedule jobs closely or could, I imagine, shut them down. But I'm not in that space either so this is merely conjecture.

* I am sure there are corner cases where the load is uniform, but they are by definition few.

large datacenters have hardware orchestration systems that let them turn off unused machines. There really is no reason to have lots of machines on but unused. At least, that is not a significant enough event to be a determining factor in hardware purchasing.

If you are the one paying the electricity bills for that datacenter, then yes, it probably matters to you a lot. If you are just renting a server from aws or gcp, it probably matters less. Although, I assume costs born from idle inefficiency will probably be passed to the customer...

I would expect most high-end servers in data centers to sit idle most of the time? Do you know of any data on this?

Most servers are doing things for human beings, and we have irregular schedules. Standard rule of thumb is that you plan for a peak capacity of 10x average. A datacenter that doesn't have significant idle capacity is one that's some kind of weird special purpose thing like a mining facility.

That's true, but I would expect that most idle servers are turned off and they use Wake-on-LAN to become active when there is work to do.

Just a few servers could be kept idle, not off, to enable a sub-second start-up time for some new work.

Servers are very expensive. It is not cost efficient to buy a server and let it sit idle or turn it off.

Even purchased on the used market, a decent server could cost $4000; if you spread that cost across 4 years, that's $80ish per month just to pay for the server, which is probably around your power / rent costs.

The costs are more stark if you're buying new, where it is easy to sped $10-30k for a decent spec server.

Don't buy a server to leave it turned off, you're wasting money.

Certainly for bit players and corporate datacenters with utilization < 1% you'd expect the median server to just sit there. For larger (amazon, google, etc) players the economic incentives against idleness are just too great.

> For larger (amazon, google, etc) players the economic incentives against idleness are just too great.

Not all workloads are CPU-bound. Cloud providers have many servers for which the CPUs are idle most of the time, because they're disk-bound, network-bound, other-server-bound, bursty, or similar. They're going to aim to minimize the idle time, but they can't eliminate it entirely given that they have customer-defined workloads.

But if the workload is not CPU-bound then why would they care about upgrading their CPUs to more performant ones, like Ice Lake Xeons?

The workloads are determined by their customers, and customers don't always pick the exact size system they need (or there isn't always an option for the exact size system they need). The major clouds are going to upgrade and offer faster CPUs as an option, people are going to use that option, and some of their workloads will end up idling the CPU. Major cloud vendors almost certainly have statistics for "here's how much idle time we have, so here's approximately how much we'd save with lower power consumption on idle".

If the workload is PCIe I/O bound, upgrading to PCIe Gen4 would be a huge benefit.

If it has 80% of the performance, it will still be competitive at 80% of the price.

This sounds like a dangerous assumption to make. I would expect that needing 25% more machines for the same performance would be a non-starter for many potential customers.

If they still cost the same in the end, it’s still a tie. If you manage 100,000 machines, managing 125,000 is not a huge leap.

Well, potentially 25000 machines more to fix

Just factor that in the price.

A bit off topic from the server CPU discussion, but I was curious how well AMD is advancing idle power consumption.

For example, the Ryzen 3000 desktop chips seemed to have the issue[0], but the same Zen 2 cores seem to have found some improvements in the Ryzen 4000 mobile chips[1].

I didn't want to just rely on Reddit forum comments, so I found this measure of the Ryzen 3600[2].

> When one thread is active, it sits at 12.8 W, but as we ramp up the cores, we get to 11.2 W per core. The non-core part of the processor, such as the IO chip, the DRAM channels and the PCIe lanes, even at idle still consume around 12-18 W in the system.

My interpretation was expect ~12 W or more idle consumption (just from the CPU package), but I'm not sure I understand it correctly.

I couldn't find the same information for Ryzen 4000 laptops, but the same APU is tested in a NUC, where the total system draw (at the wall) at idle was about 10-11 W, still nearly double that of a Core i7 U-series NUC[3], but certainly lower than that of just the CPU package in the Ryzen 3600.

Anecdotally, my 45W Ryzen 7 4800H laptop with 15.6" 1080p screen lasts about 4 hours on 80% of the 60Wh battery with 95% brightness, doing various non-intensive tasks. Though I don't know how well the battery holds up on complete non-use standby.

[0] https://old.reddit.com/r/AMDHelp/comments/cfm1xa/why_is_ryze...

[1] https://old.reddit.com/r/Amd/comments/haq4fg/the_idle_power_...

[2] https://www.anandtech.com/show/15787/amd-ryzen-5-3600-review...

[3] https://www.anandtech.com/show/16236/asrock-4x4-box4800u-ren...

Ryzen 3000 desktop processors use a chiplet design, with the IO die built on an older process than the processor dies. Ryzen 4000 mobile processors are monolithic dies, so they don't have the extra power of the inter-chiplet connections and they're entirely 7nm parts instead of a mix of 7nm and 14nm.

> I couldn't find the same information for Ryzen 4000 laptops

I measured an Asus Mini PC PN50 with a Ryzen 4500U. The idle power usage was 8.5 Watt for the system. This with 32GB of memory and a SATA SSD installed. It would be nice if it was lower than this, but it isn't too bad. Interestingly the machine used 1.2 Watt while off after it wasn't on power, 0.5 Watt after starting up and shutting it down.

Recently noticed some people focussing on low power but powerful 24/7 home "servers". Systems that are on 24/7, but often idle. One system used around 4.5 Watt in idle. The "brick" / power adapter often uses too much power, even when everything is off.

It's worth noting the difference in tone from the bulk of the discussion in this thread.

Objectively, Ice Lake "isn't too bad" in comparison with Zen 3 either. "It would be nice if it" drew a 20% less power at load, but in practice you could use either product without trouble.

In order to satisfy their wafer supply agreement with Global Foundries, AMD ships a GF fabbed I/O die in all their volume processors. I'm pretty sure that once they have met the terms required to exit that wafer supply agreement, we will see AMD shift that I/O die to a newer process with lower power draw.

You can't really compare die sizes of a MCM and a single die and expect to get transistor counts out of that. So much of the area of the MCM is taken up by all the separate phys to communicate between the chiplets and the I/O die, and the I/O die itself is on GF14nm (about equivalent to Intel 22nm) last time I checked, not a new competitive logic node.

There's probably a few more gates still on the AMD side, but it's not the half again larger that you'd expect looking at area alone.

I'm not sure that's a fair area comparison? AMD only has around 600 mm2 of expensive leading edge 7nm silicon and uses chiplets to up their yields. The rest is the connecting bits from an older and cheaper process. Intel's full size is a single monolithic die on a leading edge process.

An interesting observation, which is even more damning for Intel: moderately multicore CPUs from AMD are made in an effective and efficient way, while Intel, despite boasting about their advanced packaging technology, is slow to adopt heterogeneous chiplets and is paying the price of this technological rigidity. This is in addition to miserable failure with new processes.

Do you see getting the most of limited 7nm production capacity as "cheating"? The only fair comparison is in the resulting performance and TDP, and Intel remains seriously behind.

Do chiplets underperform compared to a monolithic die?

As others have said, in isolation and perfect manufacturing capability, it's absolutely true a monolithic die is better.

However, because each die is smaller, you can be more aggressive with your binning, potentially having a larger number of chips which you can bin into a higher frequency bucket.

Ultimately, it's a matter of trade-offs, and what's right in one place might be wrong in another (see, e.g., AMD manufacturing mobile APUs on monolithic dies versus all their desktop and server parts using chiplets).

We are seeing more of the industry moving to chiplets and similar solutions, notably Intel with their tiles.

All things being equal a chiplet design will underperform a monolithic die. But we've already seen the benchmarks on the performance of Milan so looking at chiplets versus monolithic is mostly about considering AMD's strategy and constraints rather than how the chips perform.

Pretty much any time you have signals going off chip, you lose out on both bandwidth and latency.

Yes.

It's impressive how you and parent comment copied over to/from the dupe posting verbatim.

edit oops nevermind, I see my comment was also mysteriously transported from the dupe.

Furthermore:

"At the end of the day, Ice Lake SP is a success. Performance is up, and performance per watt is up. I'm sure if we were able to test Intel's acceleration enhancements more thoroughly, we would be able to corroborate some of the results and hype that Intel wants to generate around its product. But even as a success, it’s not a traditional competitive success. The generational improvements are there and they are large, and as long as Intel is the market share leader, this should translate into upgraded systems and deployments throughout the enterprise industry. Intel is still in a tough competitive situation overall with the high quality the rest of the market is enabling."

I found it a little weird that they conclusions section didn't mention the AMD or ARM competition at all, given that the Intel chip seemed to be behind them in most of the tests.

You mean OP didn't? Yes, that's probably standard PR to focus on strengths rather than competition.

I mean the Anand piece.

The conclusions section was quoted in my post and they explicitly mention it.

"As impressive as the new Xeon 8380 is from a generational and technical stand-point, what really matters at the end of the day is how it fares up to the competition. I’ll be blunt here; nobody really expected the new ICL-SP parts to beat AMD or the new Arm competition – and it didn’t. "

>This sounds about right for a company fraught with so many process problems lately

Publicly the problems have been lately, but the things that caused these problems have happened much further back.

I'm cautiously bullish on Intel. From what I gather, Intel is in a much better place internally. They have much better focus, there is less infighting, its more engineering then sales lead, they have some very good people and they are no longer complacent. It will however take years before this is becomes visible from the outside.

Given the demand for CPUs and the competitions inability to deliver, I think intel will do OK even if they are no ones first choice of CPU vendor, while they try to catch up.

It is certainly good enough to compete, prioritising Fab capacity to Server unit and lock in those important ( Swaying ) deals from clients. Sales and Marketing work their connection along with software tools that HPC markets needs and AFAIK is still far ahead of AMD.

And I can bet those prices have lots of room for special discount to clients. Since RAM and NAND Storage dominate the cost of server, the difference of Intel and AMD shrinks rapidly in the grand scheme of things, giving Intel a chance to fight. And there is something not mentioned enough, the importance of PCI-E 4.0 Support.

I wanted to rant about AMD, but I guess there is not much point. ARM is coming.