someguy2026
Re:Show HN: Llama 3.1 70B on a single RTX 3090 via NVMe-to-GPU bypassi...
on Saturday February 21, 2026 @07:17PM (#47106600)
DRAM speeds is one thing, but you should also account for the data rate of the PCIe bus (and/or VRAM speed). But yes, holding it "lukewarm" in DRAM rather than on NVMe storage is obviously faster.
Re:Show HN: Llama 3.1 70B on a single RTX 3090 via NVMe-to-GPU bypassi...
on Saturday February 21, 2026 @06:36PM (#47106269)
My impression is that that is limited to assets and really needs to fit into the DirectX framework. From what I can tell, the gpu-nvme-direct is mostly similar to https://github.com/enfiskutensykkel/ssd-gpu-dma and https://github.com/ZaidQureshi/bam