Re:Nvidia deliberately kept GPU prices high through circular...
by searealist on Saturday November 14, 2026 @08:57PM
(#50503237)
You would need several RTX 6000 to run the large open weight models, and even then, the price of electricity would probably exceed API prices.
[ Parent
| on HN ]
Re:Nvidia deliberately kept GPU prices high through circular...
by hadlock on Saturday November 14, 2026 @09:45PM
(#50503487)
MoE next token prediction is higher than 80%; you can store the other experts in RAM and the engrams can be streamed off very mediocre nvme. It's been this way since September. I'm running it at home (although heavily quantized, there are limits) on 48gb ram and 12gb gpu. We run it at work on the 96GB RTX at a much better quant.
See also: https://github.com/niko1221/strata
Re:Nvidia deliberately kept GPU prices high through circular...
by searealist on Sunday November 15, 2026 @02:15AM
(#50504758)
You aren't hitting almost 2x decode with speculation on a MoE model, but it's irrelevant. Electricity will cost most for most people than API will, even with free hardware. And 4-bit qwen next will be a major competitive disadvantage compared to larger models.