News for hackers. Stuff that matters.
Re:Nvidia deliberately kept GPU prices high through circular... by PaulKeeble on Sunday November 15, 2026 @07:32AM (#50506474)
With Strata and the other inference engines designed for these moderate sized MOE models its more about RAM than it is the GPU. Qwen Flash Next will run on a fairly typical 12-16GB consumer card at 50 tok/s but you need 64GB to hold the weights of the experts in the model with NGRAM coming off the SSD. A blackwell GPU will run it faster but you can get a good experience with Qwen flash next at 2-4 bit quant mostly on RAM.
Re:Nvidia deliberately kept GPU prices high through circular... by hadlock on Sunday November 15, 2026 @02:33PM (#50509903)
Agreed, almost nobody needs the ability to run 60 gpu workloads concurrently, unfortunately nvidia refuses to sell a competing product with 64, 96, 128gb vram