News for hackers. Stuff that matters.
Re:Nvidia deliberately kept GPU prices high through circular... by hadlock on Saturday November 14, 2026 @09:45PM (#50503487)
MoE next token prediction is higher than 80%; you can store the other experts in RAM and the engrams can be streamed off very mediocre nvme. It's been this way since September. I'm running it at home (although heavily quantized, there are limits) on 48gb ram and 12gb gpu. We run it at work on the 96GB RTX at a much better quant.

See also: https://github.com/niko1221/strata

Re:Nvidia deliberately kept GPU prices high through circular... by searealist on Sunday November 15, 2026 @02:15AM (#50504758)
You aren't hitting almost 2x decode with speculation on a MoE model, but it's irrelevant. Electricity will cost most for most people than API will, even with free hardware. And 4-bit qwen next will be a major competitive disadvantage compared to larger models.