News for hackers. Stuff that matters.

A comment on The "high-level CPU" challenge (2008). Parent comment

Re:The "high-level CPU" challenge (2008) by mikewarot on Tuesday August 12, 2025 @04:51PM (#44881684)
The reason they need terabytes per second is they're constantly loading weights and data into and then back out of multiply accumulator chips.

If you program hardware to just do the multiply/accumulate with the weights built in, you can then reduce the bandwidth required to just putting data into a "layer" and getting the results out, a much, MUCH lower amount of data in most cases.

[ Parent | on HN ]
Re:The "high-level CPU" challenge (2008) by thrtythreeforty on Tuesday August 12, 2025 @07:03PM (#44882782)
Doesn't this presume that you can fit the model's weights into the SRAMs of a single chip, or of multiple chips that you plan to connect together? A big reason HBM is a thing is because it's much, much denser than SRAM.
[ Parent | on HN ]