News for hackers. Stuff that matters.
Re:Show HN: Llama 3.1 70B on a single RTX 3090 via NVMe-to-G... by xaskasdf on Saturday February 21, 2026 @10:30PM (#47107845)
Actually this idea was fueled by those since I went to check if there was anything near to what I wanted to achieve, pretty useful tho
Re:Show HN: Llama 3.1 70B on a single RTX 3090 via NVMe-to-G... by jonassm on Sunday February 22, 2026 @04:22AM (#47109603)
nvmlib/ssd-gpu-dma and BaM (based on the same code base) are pretty cool as they allow you to initiate disk reads/writes directly from a CUDA kernel (so not only reading/writing directly to GPU memory but also allowing the GPU to initiate IO on its own). Sometimes called GPU-initiated I/O or accelerator-initiated I/O.