Re:The Future of Memory
by ilaksh on Sunday January 21, 2024 @10:37AM
(#39079371)
I wonder if it's possible to design the next AI systems along with the hardware at the same time. For example, maybe by focusing on more approaches like mixture of experts or similar, there are ways to keep much of the data close to the cores that operate on it.
[ Parent
| on HN ]
Re:The Future of Memory
by CyberDildonics on Sunday January 21, 2024 @02:26PM
(#39081980)
That's called CPU cache. It doesn't require "mixture of experts" (whatever that would mean) it just needs transistors for SRAM.
That's one example of the more general category of what I am talking about. But I was trying to get just a little more specific.
Re:The Future of Memory
by CyberDildonics on Sunday January 21, 2024 @05:31PM
(#39083755)
Can you give another example and explain how "mixture of experts" gets data closer to a CPU?
I'm talking about GPUs and don't know the details very well. It was a rough idea.
Re:The Future of Memory
by CyberDildonics on Monday January 22, 2024 @09:32AM
(#39089897)
the more general category of what I am talking about. But I was trying to get just a little more specific
Can you get more specific then? Can you give any details or any overview? There must have been some information that led you to post this originally, can you link it?
Re:The Future of Memory
by yazaddaruvala on Tuesday January 23, 2024 @03:02AM
(#39100695)
There are many possible future architectural changes that could help. Some are more feasible than others but all require fundamental advancement to be useful and cost effective.
Some of them include:
- GPU - CPU shared memory (already possible just not yet the standard)
- Higher DRAM bandwidth (already possible, just not yet a priority)
- system on chip FPGAs (always possible just very expensive to fit “AI models”)
- SOC NVM. Ideally even NVM on the same wafer as the GPU and CPU (possible today but requires a lot of work on the yield. NVM would take up a lot of real estate that could ruin yield).
- analog circuits
- new semi-conductors / photonics
- memristors