I'm confused. According to other comment: https://news.ycombinator.com/item?id=42859645, <= 70b DeepSeek models are just a fine tuning of Llama or Qwen? So we shouldn't take any thought of these models to actually being DeepSeek.

I think people are confusing the smaller non-DeepSeek original models (Qwen/Llama) with the 700B DeepSeek R1 model being talked about in here and that very few people can run locally.

I have also been running the 32b version on my 24GB RTX 3090.

if someone wants to run the real thing (R1) locally, someone posted their hardware specs on X. Total cost: $6,000.

[0] direct link with login https://x.com/carrigmat/status/1884244369907278106

[1] alt link without login https://threadreaderapp.com/thread/1884244369907278106.html

I am doing the same.

That's not DeepSeek, it's a Qwen or Llama model distilled from DeepSeek. Not the same thing at all.

I run the 32b parameter model also just fine on our 4x H100 rig :) It's good enough for embedding, our use-case.

I'm not sure if $200k of hardware fits the consumer level