Isn’t this what Apple is doing with their M-chips?

Apple is using HBM2 ([1]) memory, pioneered by AMD for their GPUs. Essentially, yes - it's that, placing memory really close to compute and making a lot of channels for higher bandwidth.

1. https://en.wikipedia.org/wiki/High_Bandwidth_Memory

They use commodity LPDDR4X or LPDDR5 depending on SKU, for M1 and M2.

Thanks! I missed that.

uh no they’re not lmao. unless you’re talking about very particular intel machines no one cares about anymore.

HBM is RAM chip stacked on CPU chip, not an integral CPU+RAM on a single die.

No, in classic HBM multiple RAM dies are stacked on top of each other, but they sit next to the CPU, not on top. Edit: well typically it would be a GPU not a CPU.

Apple marketing made it sound like they were doing something special with RAM on their M socs. Really they mainly improved packaging form factor with run of the mill LPDDR5. It's questionable if this really does much for performance as the M socs still have worse memory latency than Intel and AMD chips.

Thank you. I indeed fell for Apple marketing.

The best silicon process for DRAM is significantly different than for CPU & logic. So it's not practical to combine them on one die -- you'd have to compromise the speed or density of one or the other.

But the chiplet approach works, and nVidia and Apple are both shipping it. I dunno what's going on at Intel. Hopefully they'll pull themselves together.

Their new Xeons have HBM in the package. It's even possible to build a workable workstation without any DRAM slots with those (provided you can get rid of 300W of heat but, for a workstation, using all cores at full speed should be a relatively rare occurrence).

> using all cores at full speed should be a relatively rare occurrence

Huh? Sustained all-core load is why I have a workstation in the first place, it's for anything that would be annoying to run on my laptop. I'm sure different people have different workloads, but I don't think I'm an outlier here.

> Huh? Sustained all-core load is why I have a workstation in the first place

At least mine spends most of the time waiting for my keystrokes. It does do background tasks, of course, and some heavy lifting sometimes (when I have to wait for them instead of the other way around) but it doesn't spend the day doing the heavy lifting because that'd make the machine much less responsive (and much noisier).

OTOH, if I had a 56-core 112-thread monster, it'd probably be perfectly fine to do the heavy lifting AND the interactive work at the same time. Worse case scenario would need to set core affinity so that the heavy lifting doesn't step over the cores running my GUI.

Apple is not shipping any chiplet designs. Its ordinary POP ram and a ton of marketing.

Some reasons that I can think of:

- Heat generation (harder to cool)

- Trade offs don't justify the pain

CMOS and DRAM are completely different processes and cannot be integrated on the same wafer. This has been tried many times unsuccessfully. This is why you see highly integrated packaging solutions like chiplets. These solutions integrate CMOS and DRAM in the same package, not on the same die.

Chiplets are also used for that, sure (stuff like radios are occasionally separate chiplet made in different process), but I think main push is smaller chips = higher yields

Thanks, this is what I'd love to understand better.

At the highest level isn't it all EUV? Why can't the laser print both patterns onto the wafer?

A CMOS chip doesn't contain any capacitors. DRAM is made of capacitors, and thus has a radically different structure.

> I’m annually shocked we don’t have DDR5 built directly into the processors same wafer.

Sapphire Rapids has models with 64GB of "High Bandwidth Memory".

> and I’d really appreciate leaving “upgradability” out of this conversation

It's not upgradability any more. That means you buy the machine, run out of ram for your working set 2 years later, add more ram.

Now Apple refuses to sell you machines with enough ram from the start.

Would you like your preferred server vendor to do the same?