Yeah, but it's not enough. The whole chip doesn't have enough SRAM to hold a tenth of GPT-3, let alone GPT-4. These are the kinds of models you would hope a chip like this would help with, but as soon as the model gets too big to fit in the SRAM then the advantage goes way down. And I think the models of the future will continue to get much, much bigger.

There are a few companies doing analog compute-in-memory for ML. Usually on flash processes.

Maybe cerebras could fit well for that.

As for the models of the future getting much bigger - is the GPT family representative ? Are we seeing the same growth is in other domains ? And isn't there a big niche for smaller than GPT models ?

You cannot make DRAM on the same process with competitive, high-end logic, and they want to embed a lot of buffers on chip to reduce communication power use, so SRAM is the only game in town.

Intel had eDRAM (Crystal Well) for a while as the cache for their integrated Iris graphics.

From wikichip: The eDRAM silicon die is separate from the main Haswell die but is packaged together with it.

Huh, I should have done more reading.

Crystal Well was only eDRAM in the marketing sense.

There have been other, actual eDRAM technologies that can be manufactured on some logic processes, however none of them are compatible with the state of the art TSMC 7nm logic process.