One thing I wonder about is that these CPUs are just so much faster than their memory controllers. So why not add more? Why don't they have a memory controller per core, for example?

I think it mostly is for more controllers to be useful that means more PHYs and more actual pins on the package and more traces on the board and more sockets for the memory and... The space and cost of all of that is substantial (to a point, and then it just physically can't be manufactured).

Cost, and especially now, cost of memory! But you can buy AMD chips with 2 controllers (consumer), 4 (threadripper), 6 (8005), 12 (EPYC) memory channels, so in effect you can. Next generation Zen 6 Epyc has 16 channels. Roughly you get about 1 channel per chiplet, but you csn adjust if you need more memory bandwidth by buying fewer chiplets, eg there is an 8 core EPYC if you want to optimisise for bandwidth. But many applications are not bandwidth bound so this isn't the right choice, eg they are IO bound, rarely compute bound.

More memory controllers would add more bandwidth. That can help, but few workloads are bottlenecked by memory bandwidth. What's really needed is less memory latency, which has not improved as much as bandwidth.

Strata is!

"few workloads are bottlenecked by memory bandwidth" becomes a self-fulfilling prophecy at a certain point. Especially when companies are marketing AI stuff but still sticking with 128 bit memory busses for their mainstream models. Except for Apple, the only company competing on this metric.

Well, in Venice they now have 16 memory controllers, 4 more than last generation.

It's not always trivial to saturate available memory bandwidth. You may need to use more than one cpu, or you need to carefully consider the topology of the chip you're working with (see lstopo). For example, what if the L3 cache is actually split in two [0]? In such cases, adding more bandwidth doesn't do much for you.

[0] https://en.algorithmica.org/hpc/cpu-cache/sharing/#saturatin...

That doesn't help memory latency and most of the time a cpu is waiting it's because of latency, not bandwidth.

Nine women cannot make a baby in one month.

128 cores x 288 pins per DIMM = 36864 pins on your socket, and all the motherboard traces. Good luck with that...

Why would each core need a direct connection to the DIMMs?

Memory controllers are pretty much limited by external pinout.

Zen 4 and newer consumer chips have 4 memory controllers (for legacy reasons shown as 2) which handle 32bit wide LPDDR5 or DDR5 interfaces (2x per dimm for DDR5)

The AM5 (desktop) socket has in addition to the 4x 32bit memory lanes, 28 PCI-E lanes (generally 16x for GPU, two 4x general purpose, one 4x to chipset), 4x combined DP/HDMI outputs, USB 2.0 port, 4 USB 3.2 Gen 2 ports. And that's just the major buses, there's ton of other components exposed (simpler IO, sound, power)

All of that counts to staggering 1718 pins - anyone remember when moving dual-channel K8 from 940 pin to 939 allowed removal of whole layer from motherboard PCB, making them way cheaper?

The high-end performance server socket from AMD has 6096 pins, significant chunk of it being the 24x 32bit DDR5 interfaces.

So unfortunately it's very much not easy to add more memory, and that's before considering issues with routing it on motherboard even if you accept changing shapes

I have no real idea of how much a "pin" costs though; it seems There was a brief period of affordable threadripper, but then product segmentation made that not great for amd...

Perhaps it's better to stick on am5 for a while, but it's not clear if this number of pins is somehow cost effective, or if it's just enough to beat Intel.

The real issue is that the cost is not really on AMD - the cost is on motherboard vendors - more complex routing, thicker PCBs, etc. - all adds up.

A pin on underside of CPU is reasonably cheaper especially with how they've got a sea of wires working for the humongous models replacing previous serdes-based approach

> There was a brief period of affordable threadripper, but then product segmentation made that not great for amd...

The thing they're doing which is completely insane is using different sockets for Threadripper and Epyc.

The primary difference between them isn't the price -- they're both expensive -- it's that Epyc uses the power budget to reach higher core counts and Threadripper uses it to hit higher clock speeds. So for example, Zen5 Threadripper 9975WX has 32 cores with a 4 GHz base clock and 5.4 GHz boost clock. There is no equivalent Zen5 Epyc, the most similar is 9375F which is 3.8GHz/4.8GHz. Conversely, Epyc 9755 has 128 Zen5 cores and Epyc 9965 has 192 Zen5c cores and there is no Threadripper with that many cores.

The reason this matters is that the system boards themselves are pricey, not just the CPUs, but as the platform starts to age you start to get a lot of old servers on the used market that originally had a bottom-end Epyc in them. Which you'd then like to pull and replace with a Threadripper -- i.e. pay them more money to buy a new Threadripper -- except that they inexplicably decided to screw you and themselves by using a different socket.

I was going to be sad than Zen 6c was divergent, but they increased a bunch of OoO structures on Zen 6c? To absorb more potential latency, since the memory is going to be heavier loaded? Ok, that I can appreciate!

It's absolutely wild that adjusting some l3 and using a smaller node goes from 12 to 32 cores for basically the same chip.

Rumor mill says they are going to use costlier interconnect to drive down CCD -> CCD latency in Zen 7+.

I think it's less there because of differences in the memory subsystem, but more because clock targets are lower so they can afford more. The structures they grew cost basically nothing in die area, but make timing tighter. It's possible they'd like them to be bigger for the non-c core too, but can't fit them in their timing constraints.

The unified L3 per CCD really helps on some key desktop loads (especially games). Most server throughput loads partition nicely into smaller partitions, but most games are hurt so bad by the latency penalty of crossing into a different partition that AMD's game drivers park the second CCD on their 2-CCD CPUs when they detect a game running. So long as games are an important target for them, I suspect they will not partition the CCD again. (It, of course, used to be parititioned for Zen 1 and 2.)

The instruction level parallelism in the 2-ahead branch prediction is super cool stuff, I know zen5 had this feature but it's kind of crazy that memory is so slow relative to the cores that this complexity is beneficial and worthwhile.

[delayed]

SIMD stuff is also generally pretty big part of the chip, tho of course dwarfed by cache

It has always been like this, and it’s amazing how branch prediction and speculative execution work. CPUs are so much more complex than the simple mental model you first learn.

> It has always been like this

Nah. I recall putting multiplication tables in RAM because it was faster to read the tables than doing the multiplication.

RAM used to be relatively fast. These days it's dog slow unless you're streaming.

It is hilarious that the SPEC2026 graph does not include apple's P-cores. And no, not because they are ARM and not x86, since Graviton is included. It is because their integer & FP scores of 9.8 & 13 respectively absolutely wipe out everything else on this chart.

I mean, this is a review of the server grade parts. Apple is not exactly selling m chips to hyperscalers right?

Different story when you're talking about the context of a desktop or mobile.

The same graph has a bar for the Ryzen 9 9950X, though.

Ryzen 9950X is basically the same as an Epyc 4565P, except you can buy the Ryzen in a retail box.

1. Different market

2. Can't run Linux on that M5/M6 can we?

1. so is Ryzen 9 9950X

1. 9950X isn't on the chart.

The 9950X3D2 was already tested. And was included as comparison point with the fastest possible version of its predecessor (Zen 5).

Some evidence that contextless numbers are meaningless is that your source, David Huang, does include the 9950X but in his results it scores much higher than everything on C&C's chart too (7.25/11.4)

Regular Ryzen parts are still reasonably relevant for server use! They support ECC, so there's nothing immediately disqualifying them - see the rebranded "EPYC" 4005 series, for example.

A company like Hetzner isn't going to care an awful lot whether they deploy one big SP5 server or four small AM5 ones for the VM instances they rent out: the only thing that matters is TCO per performance.

Want to stick four NVMe SSDs, a 25Gbit NIC, 256GB of ECC RAM, and some OOB module on a Ryzen board? No problem whatsoever. Want to do the same with an Apple CPU? Ehhh, that's going to be slightly more challenging...

Linux is now (barely) working on M5/M6.

if you're interested in a fair comparison here's one https://openbenchmarking.org/test/pts/stockfish

Interesting, but seems to lack single-core benchmarks.

it's not year 2006 anymore

I’m sorry, I wasn’t aware that Amdahl’s law has been cancelled. I will brush up on my computing theory.

Workloads with legitimate serialization dependencies rather than just bad code are pretty rare. Of course, there is still a lot of bad code out there, but it's not Amdahl's law stopping them from doing better.

So this benchmark lists the following:

> AMD Ryzen 9 9950X 16-Core 63rd 4 40132379 +/- 2530724

> Apple M5 Pro 52nd 5 22108587 +/- 260160

I have absolutely no idea what this is benchmarking or how, but is there any reason not to just read this as the "consumer/low core ryzen" is twice as fast as the m5 pro?

> I have absolutely no idea what this is benchmarking or how

It's the best chess engine used by millions of people and part of SPEC2026, natively compiled

Stockfish parallelizes well and makes heavy use of vector operations, and the Zen5 core has AVX-512 with 2x 512bit EU per core and 16 cores, while the Apple still only uses 128-bit wide SIMD, with the 6 M5 "Super" cores having 4x128 EUs and the 12 "Performance" cores only having 2x128 EUs.

The M5 Pro is doing shockingly well given the massive disparity in execution resources. But still, it's optimized for web/desktop-like loads, you shouldn't really run heavy number crunching on that CPU.

Is there any sane way to generalize these results?

I don't think so. The workloads are simply too different to generalize.

They also mention using SSE/AVX on x86 and NEON on ARM, but there's zero mention of SVE, SME, or AMX which makes me wonder if they aren't compiling with those options (if so, it would be a serious disadvantage).

Apple cores do not implement SVE, SME is not suitable for Stockfish (it is optimized for matrix-matric with large batch size, stockfish is all matrix-vector with batch size of 1), and AMX has the same problem.

You don't get good performance out of this kind of code by just compiling with different options, the compute kernels must be tuned to fit the architecture, and nothing on the M-series but NEON fits the bill.

The single-threaded results of the Apple cores are excellent, but the multi-threaded results per socket of a Zen 6 absolutely wipe out any Apple CPU.

The Apple CPUs may be great for a personal computer, but they are not competitive with a server CPU.

Moreover, the SPEC floating-point results are not really representative for the floating-point performance of Zen 5 and Zen 6 that can be reached with optimized AVX-512 programs, which is much greater.

Apple could absolutely make a server CPU using their E-cores that would wipe the floor with Zen6 in both perf and perf/watt. Apple isn't in the server hardware business though which would make it a pointless piece of engineering.

Sure

What looks like a lot of small balanced improvements to the core itself. Four vector loads per cycle at 256b of less, I wonder if the gather performance is finally acceptable?

The real star looks like the c cores though. 32 C cores per chiplet with increased l3 cache is impressive

> AMD has reduced the number of integer multiply units from three to two, with PDEP and PEXT throughput also falling from three instructions per cycle to two.

Are PDEP and PEXT using the same (parts of) circuits as multiply (they are very different operations...), or are this just two independent drops of circuitry and the wording is just a bit unclear?

They are unlikely to share circuits with the multipliers, but the 2 of the 6 execution units that include multipliers must be identical, to simplify layout, and the PDEP/PEXT circuits are allocated to these 2 units, like the multipliers.

IIRC, PDEP/PEXT are generalizations of shifter networks.

A multiplier also contains a generalized shifter network.

Multiplying by 2^n = shift by n.

32 cores is a lot of cores on a CCD. Is there a there any smaller logical unit between a CCD and a core, or are you expected to wrangle 32 cores in your lowest NUMA level?

From what I've seen there is no intermediate unit. IMO this is good because it allows larger VMs that don't have to straddle L3s.

32 cores is only on the Zen 6c CCDs. There are 12 cores on the Zen 6 CCDs.

Seems like they are starting to optimize the C core architecture already. Wonder how long it'll be before it becomes its own line up and branding.

This is amusing since Intel seems to be sort of going in the other direction lately: their efficiency cores are becoming more like their performance cores with each new generation (at least in so far as they are getting wider and more capable of higher clock speeds), and there even seem to be some rumors that the two design trains may get "merged" in the future.

It's interesting to hear indeed. The 6c's improved latency hiding might be intended for use in extremely high core count Epycs, which have lots of memory channels, but not many at all per core.

The worst thing about consumer (AM5) Zen4 and Zen5 is the I/O die chiplet and memory controller on it. I've heard they're finally going to ship a new I/O chiplet in Zen6 on a much smaller node. Their AM5 memory controller when using both memory channels is terrible. Extremely hard or impossible to get above 3600 MT/s.

Of all the things I'm looking forward to on Zen6, an improved memory controller is at the top of the list.

Edit: and yes, others may say the paltry few PCIE 5.0 lanes is the worst thing about AM5, but that doesn't affect my usage much, and unfortunately I think that's a limitation of AM5 itself rather than the I/O chiplet.

(AM4 was 3600 MT/s; AM5 is 6000 MT/s.)

Reported even on AM5, you can only get 6000 Mt/s with 2 sticks, attempting to run all 4 slots on most boards won't work unless you slow them down to base JEDEC speeds.

Can confirm, had this issue on an ASRock DeskMeet X600.

Really? I'm running 4x 24GB sticks at 6000 with no issues. These were on the certified list for the motherboard.

https://valid.cpuid.com/sdyce7k2

4 x 32GB G.Skill Flare 6000CL30 are running on Asus TUF B650E with 6000CL34

AM5 is 3600MT/s for 4 sticks; check the specification of a CPU of interest to you, expand the Connectivity section. For 2 sticks it's 5600MT/s. The Ryzen 9 9950x3d officially only supports 5600 or 3600.

https://www.amd.com/en/products/processors/desktops/ryzen/90...

Obviously, you can run memory faster than the officially supported speed, including 4 DIMMS, but the Zen 5 architecture has a reputation for being finicky trying to run 4 DIMMs at higher speeds and you need a bit of care in what you pick and how you tune it.

I'm running 8000 in 2:1 on my 9800x3D. Slightly better than 6200 1:1 on here, and basically the same as 6400 1:1, but easier to dial in at 8000.

Side note: At 2ghz IF, a single Zen4/5 CCD can saturate 7200 MT DDR5, but the io die is not restricted to that, so x16 and IGPU DMA can still saturate it at 8000.

Agreed, this is likely going to change the landscape for people ekeing everything out of an AMD-based HEDT.

To take full advantage of it, folks likely will want A) a new motherboard with likely a new chipset to support the higher frequencies and B) <GULP> higher frequency DDR5.

I’d love more memory bandwidth with the limited 2-channels I have, but the 2x48GB (96GB) of “tuner memory” I bought in February 2025 I’m not even sure is manufactured anymore. I paid $440 back then, and my understanding is it’s not produced now, and commands $2500 - $3000 on the second-hand market.

It's not too hard to get above 3600 MT/s (I assume what you actually want is to get the uclk above 1800) if you bump the soc voltage, but at default voltages there's no chance. Most people overclocking don't touch the soc rail. I've done it on a few zen4 parts and supposedly there are golden samples that go above 1800 without major soc voltage increases but I've never seen evidence of one.

I'm talking about with 4 sticks, not 2.

I've never gotten 4 sticks to pass training, no matter what I change. This is a well-known limitation. Some people, with newer AGEIS on certain motherboards, have been able to get 4 sticks above 3600 MT/s but it's not possible for most people, and questionably stable even for those that have succeeded.

I've had no issues on my ASRock X870E Taichi from the very beginning. I'm running four 24 gig sticks at 6000 MT/s (Corsair, all bought before the crazy). I didn't realize people were having this many issues. These are on the certified lists from the manufacturer of the motherboard. Never had a problem. They've been working through multiple AGESA updates / multiple BIOS updates without a hitch.

X870e is not am4 though. That's ddr5 and a different memory controller.

The initial comment wasn't about AM4.

> Their existing AM5 Zen4/5 memory controller when using both memory channels is terrible. Extremely hard or impossible to get above 3600 MT/s.

I’m on an MSI x670e with 4x32 sticks, also Corsair. Never gotten training beyond 3600. I’ve recently seen people posting success stories on my same motherboard at 6000 MT thanks to newer bios AGESA. But none of that has worked for me.

Not sure how deep into it you went, but I found with 4 sticks I had to bump soc voltage, memory voltage, and loosen all my timings, especially tRFC. I can only pass training at marginally higher mem clocks with tRFC around 350ns with 4 sticks whereas 2 sticks was down near 200ns. Is it perfectly stable though... I think it's crashed once in the past month? So not perfect but it passed 12-24h of every stress test I could think of, and the one crash was near idle which probably just means my voltages need another bump.

I have a 9950X with 4x48G sticks. It is so hard to get it to boot even at 3600. But I persevere to run my local LLMs. I would love to get a new reliable chip for this same motherboard and RAM.

I wonder if this also applies to APUs like the Strix Halo?

Strix Halo has a newer IO die with twice as many memory controllers, and a better interconnect to the CPU dies. I'm not sure if the individual memory controllers are much better than on the Zen4/5 desktop IO die; I don't think I've seen a Strix Halo system built to use desktop DIMMs so testing how it handles two DIMMs per channel isn't practical.

The 9996 in this article has 16 channels for 1.6TB/s and 256 cores/512 threads. Don't know if this will translate to consumer parts.

I heard they had issues with the new consumer zen5 I/O die, so unfortunately had to revert to zen4 I/O die for the consumer zen5 CPUs

This is why I love the Ryzen 395 as it has so much more bandwidth than the typical consumer CPU has. Now I would have loved some PCIE 5.0 lanes.

The brach predictor it's the bane of programmers. It forces to waste effort to write unreadable branchless code.

X86 should support no-branch-predict-if and dont-predict-goto

Won't that just stall the pipeline on a modern superscalar core and cause even worse performance?

Yes. I don’t think this person (presumed) knows what a branch predictor does.

Select my post, right click, press search, and Google ai will explain to you basic stuff about branchless programming that you should know already.

There are use cases where branch prediction performs better and others where it performs worse. Ask an AI about branchless programming and when it can improve performance. We need both types of CPU instructions to achieve optimal performance.

You've got it the wrong way around. If the branch predictor was 100% accurate, branchless code would have no performance benefit compared to branching code.

> X86 should support no-branch-predict-if

It almost does. You can use __builtin_expect to hint if the branch is predictable - like assertions. Don't annotate conditionals if they're truly unpredictable. The branch predictor will run, but it's not expensive.

You should prefer using the standardized likely/unlikely attributes instead.

Branchless programming is a thing because those don't work, and branch prediction hurts performance on many cases