I think it mostly is for more controllers to be useful that means more PHYs and more actual pins on the package and more traces on the board and more sockets for the memory and... The space and cost of all of that is substantial (to a point, and then it just physically can't be manufactured).

Cost, and especially now, cost of memory! But you can buy AMD chips with 2 controllers (consumer), 4 (threadripper), 6 (8005), 12 (EPYC) memory channels, so in effect you can. Next generation Zen 6 Epyc has 16 channels. Roughly you get about 1 channel per chiplet, but you csn adjust if you need more memory bandwidth by buying fewer chiplets, eg there is an 8 core EPYC if you want to optimisise for bandwidth. But many applications are not bandwidth bound so this isn't the right choice, eg they are IO bound, rarely compute bound.

More memory controllers would add more bandwidth. That can help, but few workloads are bottlenecked by memory bandwidth. What's really needed is less memory latency, which has not improved as much as bandwidth.

Strata is!

"few workloads are bottlenecked by memory bandwidth" becomes a self-fulfilling prophecy at a certain point. Especially when companies are marketing AI stuff but still sticking with 128 bit memory busses for their mainstream models. Except for Apple, the only company competing on this metric.

Well, in Venice they now have 16 memory controllers, 4 more than last generation.

It's not always trivial to saturate available memory bandwidth. You may need to use more than one cpu, or you need to carefully consider the topology of the chip you're working with (see lstopo). For example, what if the L3 cache is actually split in two [0]? In such cases, adding more bandwidth doesn't do much for you.

[0] https://en.algorithmica.org/hpc/cpu-cache/sharing/#saturatin...

That doesn't help memory latency and most of the time a cpu is waiting it's because of latency, not bandwidth.

Nine women cannot make a baby in one month.

128 cores x 288 pins per DIMM = 36864 pins on your socket, and all the motherboard traces. Good luck with that...

Why would each core need a direct connection to the DIMMs?

That'd give you the best memory bandwidth. A single core is more than capable of saturating a single memory channel in certain streams of instructions.

Early AMD x86-64 chips were exactly that, and EPYC Naples (gen 1) was similar - each CCD had 2 memory channels. Later EPYCs ran the memory controllers from a separate IOD, in part because the strong NUMA topology was a bit too much for some OS's (Windows) to handle.

But these days with the very high core counts, it is infeasible for a memory channel per core.

Memory controllers are pretty much limited by external pinout.

Zen 4 and newer consumer chips have 4 memory controllers (for legacy reasons shown as 2) which handle 32bit wide LPDDR5 or DDR5 interfaces (2x per dimm for DDR5)

The AM5 (desktop) socket has in addition to the 4x 32bit memory lanes, 28 PCI-E lanes (generally 16x for GPU, two 4x general purpose, one 4x to chipset), 4x combined DP/HDMI outputs, USB 2.0 port, 4 USB 3.2 Gen 2 ports. And that's just the major buses, there's ton of other components exposed (simpler IO, sound, power)

All of that counts to staggering 1718 pins - anyone remember when moving dual-channel K8 from 940 pin to 939 allowed removal of whole layer from motherboard PCB, making them way cheaper?

The high-end performance server socket from AMD has 6096 pins, significant chunk of it being the 24x 32bit DDR5 interfaces.

So unfortunately it's very much not easy to add more memory, and that's before considering issues with routing it on motherboard even if you accept changing shapes

I have no real idea of how much a "pin" costs though; it seems There was a brief period of affordable threadripper, but then product segmentation made that not great for amd...

Perhaps it's better to stick on am5 for a while, but it's not clear if this number of pins is somehow cost effective, or if it's just enough to beat Intel.

The real issue is that the cost is not really on AMD - the cost is on motherboard vendors - more complex routing, thicker PCBs, etc. - all adds up.

A pin on underside of CPU is reasonably cheaper especially with how they've got a sea of wires working for the humongous models replacing previous serdes-based approach

> There was a brief period of affordable threadripper, but then product segmentation made that not great for amd...

The thing they're doing which is completely insane is using different sockets for Threadripper and Epyc.

The primary difference between them isn't the price -- they're both expensive -- it's that Epyc uses the power budget to reach higher core counts and Threadripper uses it to hit higher clock speeds. So for example, Zen5 Threadripper 9975WX has 32 cores with a 4 GHz base clock and 5.4 GHz boost clock. There is no equivalent Zen5 Epyc, the most similar is 9375F which is 3.8GHz/4.8GHz. Conversely, Epyc 9755 has 128 Zen5 cores and Epyc 9965 has 192 Zen5c cores and there is no Threadripper with that many cores.

The reason this matters is that the system boards themselves are pricey, not just the CPUs, but as the platform starts to age you start to get a lot of old servers on the used market that originally had a bottom-end Epyc in them. Which you'd then like to pull and replace with a Threadripper -- i.e. pay them more money to buy a new Threadripper -- except that they inexplicably decided to screw you and themselves by using a different socket.