Why did it take so long for us to get here?

Just a guess, but fabricating this can't be easy. Yield is probably higher if you have less memory per chip.

It's regular memory on separate chips.

Some possible groups of reasons: 1. Until recently RAM amount was something the end user liked to configure, so little market demand. 2. Technically, building such a large system on a chip or collection of chiplets was not possible. 3. RAM speed wasn't a bottleneck for most tasks, it was IO or CPU. LLMs changed this.

M1 came out before the LLM rush, though

Apple has always liked to integrate as much as possible on the same chip. It was only natural that they would come to this conclusion, with the improved perf the cherry on top.

Well also these chips originated in phones, where they kinda had to integrate it. And the quicker RAM and disk access are pretty nice.

The M1 is in a product segment where discrete GPUs have been gone for decades, in favor of integrated graphics that shares one pool of RAM with the CPU. The better question to ask is why Apple kept using that unified memory design even when moving up to larger chips like the M1 Max and M1 Ultra.

The GPU is built into the same physical die as the CPU.

So if you wanted to give it a second ram pool you would have to add an entire second memory interface just for the on-die GPU.

Now all you’ve done is make it more complicated, slower because now you have to move things between the two pools, and gained what exactly?

I think it was a very clear and obvious decision to make. It’s an outgrowth out of how the base chips were designed, and it turned out to be extremely handy for some things. Plus since all their modern devices now work this way that probably simplify the software.

I’m not saying it’s genius foresight, but it certainly worked out rather well. There’s nothing stopping them from supporting discreet GPUs too if they wanted to. They just clearly don’t.

I'd guess that they inherited it from the iPhone chips. It was nice and fast and also makes Apple a lot of profit as no third party RAM is possible.

They put the M1 into the desktops too

Apple debuted dedicated machine learning hardware in 2017 with the Neural Engine on iPhones. While I don’t think they predicted the LLM explosion in particular, they knew machine learning was important and they have been allowing that to influence hardware design.

Laptops have had unified memory for ten years or more. For desktops very few apps benefit from unified memory.

And game consoles that use similar parts as laptops.

The real hardware needed for artificial intelligence wasn't NVIDIA, it was a CRAY XMP from 1982 all along

WHen I was with Mirantis, I flew to Austin TX to meet a client in a non-descript multi-tenant office building...

we walked in and getting our bearings, we come upon CRAY office. WTF?!

I tried the doors, locked - and it was clearly empty... but damn did I want to steal their office door signage.

It's new for mainstream PCs to have it.

New for performance machines maybe. I remember "integrated graphics" when that meant some shitty co-processor and 16 or 32MB of semi-reserved system RAM.

Nope, it was common in 8 and 16 bit home computers, and in respect to PCs themselves graphics memory was mapped into the main memory until the arrival of 3D dedicated cards.

And even with 3D, integrated GPUs have existed for years.

Like pretty much every game console.

The CPUs with iGPUs didn't also have the memory on-chip. The Nintendo 64 did. Not sure about the old home computers, but I thought those had separate memory usually.

Of course not, because they are not designed as SOCs, the only memory on chip is cache, it doesn't change the fact the memory is one whole block shared between CPU and iGPU.

Apple does not have the memory on-chip (on the same die as the CPU) either.

It's not new for PC to block user ram upgrade

You mean the room sized super computer than sold tens of units?

Yes, but now its in my pocket.