News for hackers. Stuff that matters.
Re:Micron Introduces 128 GB DDR5-8000 RDIMMs with Monolithic... by phkahler on Monday November 13, 2023 @01:21PM (#38253337)
>> The demand for large memory capacity RDIMMs is being primarily driven by the sudden emergence of large-language models (LLMs) for generative AI and increasing CPU core counts.

I know LLMs are undergoing rapid change, but what if someone (say Apple) wanted to commit a large model to an actual masked ROM? What kind of density can be achieved on a similar process as DRAM, and what kind of interface would such a thing use?

Re:Micron Introduces 128 GB DDR5-8000 RDIMMs with Monolithic... by dragontamer on Monday November 13, 2023 @02:03PM (#38253835)
No one uses ROM anymore. Its all Flash RAM, which is denser, cheaper, lower-power in practice. And Flash is likely too slow in any case, so people will prefer DRAM.
Re:Micron Introduces 128 GB DDR5-8000 RDIMMs with Monolithic... by dist-epoch on Monday November 13, 2023 @02:11PM (#38253936)
You can store the weights in ROM.

But you also have input dependent activations. You need RAM for these.