Re:Show HN: Llama 3.1 70B on a single RTX 3090 via NVMe-to-G...
by someguy2026 on Saturday February 21, 2026 @07:17PM
(#47106600)
DRAM speeds is one thing, but you should also account for the data rate of the PCIe bus (and/or VRAM speed). But yes, holding it "lukewarm" in DRAM rather than on NVMe storage is obviously faster.
[ Parent
| on HN ]
Re:Show HN: Llama 3.1 70B on a single RTX 3090 via NVMe-to-G...
by tgrowazay on Monday February 23, 2026 @03:00PM
(#47127899)
Yes.
In general systems usually have PCIE version with bandwidth better than RAM of that system.
For example a system with DDR4 (27Gbs) usually has at least PCIE4 (32Gbs at 16x).
But you can bottleneck that by building a DDR5 (40Gbs) system with PCIE4 card.