jonassm
Re:Show HN: Llama 3.1 70B on a single RTX 3090 via NVMe-to-GPU bypassi...
on Sunday February 22, 2026 @01:08PM (#47113210)
Optanes will be good for latency, but not so much for BW which seems to be your major bottleneck if I'm not mistaken?
Re:Show HN: Llama 3.1 70B on a single RTX 3090 via NVMe-to-GPU bypassi...
on Sunday February 22, 2026 @12:37PM (#47112889)
In those systems you could probably leverage something like Nvidia SCADA or GDS directly.
Re:Show HN: Llama 3.1 70B on a single RTX 3090 via NVMe-to-GPU bypassi...
on Sunday February 22, 2026 @04:22AM (#47109603)
nvmlib/ssd-gpu-dma and BaM (based on the same code base) are pretty cool as they allow you to initiate disk reads/writes directly from a CUDA kernel (so not only reading/writing directly to GPU memory but also allowing the GPU to initiate IO on its own). Sometimes called GPU-initiated I/O or accelerator-initiated I/O.