News for hackers. Stuff that matters.

A comment on Show HN: Llama 3.1 70B on a single RTX 3090 via NVMe-to-GPU bypassing the CPU. Parent comment

Re:Show HN: Llama 3.1 70B on a single RTX 3090 via NVMe-to-G... by fabifabulous on Sunday February 22, 2026 @12:43PM (#47112964)
NVMEs are much, much slower than RAM. Especially unified/soldered RAM.
[ Parent | on HN ]
Re:Show HN: Llama 3.1 70B on a single RTX 3090 via NVMe-to-G... by lmeyerov on Monday February 23, 2026 @12:34AM (#47118490)
Bandwidth-wise, it's fun when you have a storage array instead of just 1 nvme. Then you can saturate the pcies, and go beyond what's cost effective on ram. Interesting to think of this as opening the door to 10-100T MoEs..
[ Parent | on HN ]
Re:Show HN: Llama 3.1 70B on a single RTX 3090 via NVMe-to-G... by placatedmayhem on Monday February 23, 2026 @01:28AM (#47118799)
Wasn't there a storage device some Years ago (decade plus) that was RAM strapped to a PCI-E card with the electronics to present the RAM as a storage device?
[ Parent | on HN ]