Re:Show HN: Llama 3.1 70B on a single RTX 3090 via NVMe-to-G...
by davideom0414 on Sunday February 22, 2026 @07:50AM
(#47110592)
Really interesting experiment i should have done this before
Do you have numbers on effective throughput vs PCIe theoretical bandwidth?
I’m curious whether this is primarily latency-bound or bandwidth-bound in practice
Can some tell me??
[ Parent
| on HN ]
Re:Show HN: Llama 3.1 70B on a single RTX 3090 via NVMe-to-G...
by xaskasdf on Sunday February 22, 2026 @10:03AM
(#47111553)
Actually is purely bandwidth-bound, the major bottleneck of the whole process, for me in this case, is the B450 mobo I got that's only capable of pcie3 and 1x8 in the pcie lanes for gpu instead of 1x16; so I'm capped until I get an X570 maybe. I should get around twice or triple the tok speed with that upgrade alone