News for hackers. Stuff that matters.
Re:Matrix-vector multiplication implemented in off-the-shelf... by nkurz on Sunday May 04, 2025 @11:02PM (#43891592)
Yup, and incredibly they are able to do this on standard RAM by "intentionally violating the timing parameters":

Processing-Using-DRAM (PUD) leverages the inherent analog operational characteristics of DRAM to enable highly parallel bit-serial computations directly within memory arrays. Prior research has demonstrated that commercial off-the-shelf DRAM can achieve PUD functionality without hardware modifications by intentionally violating the timing parameters.

These studies have established two fundamental PUD operations: RowCopy and majority-of-X (MAJX) (Fig. 1). The RowCopy operation facilitates data movement between different rows within a subarray by issuing a PRE command followed immediately by an ACT command before bitline precharging completes, enabling data transfer through the bitlines. This operation affects all cells along a row simultaneously, making it approximately 100 times faster than processor-mediated data movement. The MAJX operation performs a majority vote among X cells sharing the same bitline that are activated simultaneously, implemented in commercial DRAM by issuing ACT, PRE, and ACT commands in rapid succession without delays. This allows concurrent activation of 2∼32 rows. MAJX enables bit-serial computations that leverage the parallelism of subarrays with 65,536 columns, serving as the fundamental computational unit for PUD.

Re:Matrix-vector multiplication implemented in off-the-shelf... by nayuki on Sunday May 04, 2025 @11:53PM (#43891813)
This kind of low-level protocol manipulation of DRAM has some similarities to rowhammer attacks.
Re:Matrix-vector multiplication implemented in off-the-shelf... by gwern on Monday May 05, 2025 @10:43AM (#43895669)
Can it be used to covertly run computations invisible to the OS or CPU?
Re:Matrix-vector multiplication implemented in off-the-shelf... by wtallis on Monday May 05, 2025 @11:24AM (#43896074)
Only to the same extent that any other co-processor add-in card can do stuff that's not observable by the CPU. Your CPU's RAM is managed by the CPU's memory controller hardware, and that memory controller does not give software the ability to issue individual DRAM commands like precharge. This research uses a memory controller implemented on a FPGA, talking to its own pool of RAM.
Re:Matrix-vector multiplication implemented in off-the-shelf... by nsteel on Monday May 05, 2025 @11:14AM (#43895980)
This research requires a custom memory controller that's doing "weird" things, the CPU isn't really getting involved here. It's very different compared to row hammer in my opinion. If you have a custom memory controller then I think all bets are off.
Re:Matrix-vector multiplication implemented in off-the-shelf... by elcritch on Monday May 05, 2025 @01:08AM (#43892085)
I hope Micron or another commercial player builds a product on this!
Re:Matrix-vector multiplication implemented in off-the-shelf... by tamlin on Monday May 05, 2025 @03:17AM (#43892622)
Samsung and SK-Hynix have had specs and papers for a few years already for HBM and GDDR. e.g.

* https://www.servethehome.com/sk-hynix-ai-memory-at-hot-chips... * https://www.servethehome.com/samsung-processing-in-memory-te...

Not sure anyone has started using it in production.

Re:Matrix-vector multiplication implemented in off-the-shelf... by nsteel on Monday May 05, 2025 @11:16AM (#43895998)
And as mentioned in a comment elsewhere, LPDDR6-PIM is coming along too https://wccftech.com/samsung-collaborates-with-sk-hynix-in-p...

We'll see that before anything built around HBM or GDDR.