Re:I built a dual RTX 3090 rig for local AI in 2025 (and les...
by Tepix on Tuesday September 23, 2025 @02:24PM
(#45350912)
In general, if all you do is inference with a model that’s in VRAM, you’re right. OTOH it’s simply a matter of picking the right mainboard. If you have one of those sweet new MoE models that won‘t completely fit in your VRAM, offloading means you want PCIe bandwidth, because it will be a bottleneck.
Also swapping between LLMs will be faster.
[ Parent
| on HN ]
There are no comments on this comment yet.