Re:DeepSeek V4–almost on the frontier, a fraction of the price
by ryan-a on Saturday May 02, 2026 @05:45PM
(#47990885)
You don't need to run the model locally if you don't care about sharing your data. Personally I am happy to share data with Kimi or Deepseek if it means we get better OSS models. For private stuff though local is king
Re:DeepSeek V4–almost on the frontier, a fraction of the price
by skeledrew on Saturday May 02, 2026 @10:47PM
(#47992827)
It'll be a while yet before open models that're good enough will be viable for local use. Heck I've been trying to use the Qwen 3.5 39B A3B on my system, which is modest but no slouch, and have only been able to get ~4.5 tok/s after optimization, and it really runs my system red (fans instantly go crazy). It's just not practical for serious work.
Re:DeepSeek V4–almost on the frontier, a fraction of the price
by Zambyte on Sunday May 03, 2026 @08:06AM
(#47996113)
I've been using Qwen 3.5 and then 3.6 27b Q4 on Ollama with a single 7900 XTX with the codex cli, and I have been blown away by how genuinely useful it is. I've been able to ask it to do long, multi step problems, and it's able to do things that would have likely taken me days to iron out in a matter of hours, or even minutes sometimes.
I get about 30 tok/s, which is far from blazing, but given the capability it has it is absolutely viable for accelerating my work.
While running them locally presently doesn't make sense economically, you don't need to run them locally to address this issue. There is a lot of competition in hosting open models and you have a variety of services to choose from. Run the open models now, reward that ecosystem instead of continuing to reward closed systems that dreams of rent-seeking.