News for hackers. Stuff that matters.
Re:Show HN: Can I run this LLM? (locally) by do_not_redeem on Saturday March 08, 2025 @07:35PM (#43304969)
> Can I Run DeepSeek R1

> Yes, you can run this model! Your system has sufficient resources (16GB RAM, 12GB VRAM) to run the smaller distilled version (likely 7B parameters or less) of this model.

Last I checked DeepSeek R1 was a 671B model, not a 7B model. Was this site made with AI?

Re:Show HN: Can I run this LLM? (locally) by jsheard on Saturday March 08, 2025 @07:49PM (#43305058)
> Was this site made with AI?

OP said they "vibe coded" it, so yes.

https://en.m.wikipedia.org/wiki/Vibe_coding

Re:Show HN: Can I run this LLM? (locally) by kennysoona on Saturday March 08, 2025 @10:56PM (#43306052)
Goodness. I love getting older and see the ridiculousness of the next generation.
Re:Show HN: Can I run this LLM? (locally) by reaperman on Saturday March 08, 2025 @07:41PM (#43305000)
It says “smaller distilled model” in your own quote which, generously, also implies quantized.

Here[0] are some 1.5B and 8B distilled+quantized derivatives of DeepSeek. However, I don’t find a 7B model, that seems totally made-up from whole cloth. Also, I personally wouldn’t call this 8B model “DeepSeek”.

0: https://www.reddit.com/r/LocalLLaMA/comments/1iskrsp/quantiz...

Re:Show HN: Can I run this LLM? (locally) by monocasa on Saturday March 08, 2025 @07:52PM (#43305082)
Is it maybe because DeepSeek is a MoE and doesn't require all parameters for a given token?

That's not ideal from a token throughput perspective, but I can see min working set of weight memory gains if you can load pieces into vram for each token.

Re:Show HN: Can I run this LLM? (locally) by throwaway314155 on Saturday March 08, 2025 @07:58PM (#43305129)
It still wouldn't fit in 16 GB memory. Further there's too much swapping going on with MoE models to move expert layers to and from gpu without bottlenecks.
Re:Show HN: Can I run this LLM? (locally) by sudohackthenews on Saturday March 08, 2025 @07:37PM (#43304986)
> > smaller distilled version

Not technically the full R1 model, it’s talking about the distillations where Deepseek trained Qwen and Llama models based on R1 output

Re:Show HN: Can I run this LLM? (locally) by do_not_redeem on Saturday March 08, 2025 @07:42PM (#43305010)
Then how about DeepSeek R1 GGUF:

> Yes, you can run this model! Your system has sufficient resources (16GB RAM, 12GB VRAM) to run this model.

No mention of distillations. This was definitely either made by AI, or someone picking numbers for the models totally at random.

Re:Show HN: Can I run this LLM? (locally) by sudohackthenews on Saturday March 08, 2025 @08:17PM (#43305242)
Ok yeah that’s just weird
Re:Show HN: Can I run this LLM? (locally) by wbakst on Saturday March 08, 2025 @08:51PM (#43305407)
lol words out of my mouth