News for hackers. Stuff that matters.
Re:Show HN: Llama 3.1 70B on a single RTX 3090 via NVMe-to-G... by throwaway2027 on Saturday February 21, 2026 @06:14PM (#47106010)
Didn't DirectX add an API for loading assets directly to GPU memory? Would that work?
Re:Show HN: Llama 3.1 70B on a single RTX 3090 via NVMe-to-G... by someguy2026 on Saturday February 21, 2026 @06:36PM (#47106269)
My impression is that that is limited to assets and really needs to fit into the DirectX framework. From what I can tell, the gpu-nvme-direct is mostly similar to https://github.com/enfiskutensykkel/ssd-gpu-dma and https://github.com/ZaidQureshi/bam
Re:Show HN: Llama 3.1 70B on a single RTX 3090 via NVMe-to-G... by xaskasdf on Saturday February 21, 2026 @10:30PM (#47107845)
Actually this idea was fueled by those since I went to check if there was anything near to what I wanted to achieve, pretty useful tho
Re:Show HN: Llama 3.1 70B on a single RTX 3090 via NVMe-to-G... by jonassm on Sunday February 22, 2026 @04:22AM (#47109603)
nvmlib/ssd-gpu-dma and BaM (based on the same code base) are pretty cool as they allow you to initiate disk reads/writes directly from a CUDA kernel (so not only reading/writing directly to GPU memory but also allowing the GPU to initiate IO on its own). Sometimes called GPU-initiated I/O or accelerator-initiated I/O.