DeepSeek proves the future of LLMs is open-source

Re:DeepSeek proves the future of LLMs is open-source by marcodiego on Wednesday January 29, 2025 @11:10AM (#42866759)

Isn't DeepSeek simple/small enough you can run it locally?

[ Parent | on HN ]

Re:DeepSeek proves the future of LLMs is open-source by zbendefy on Wednesday January 29, 2025 @11:44AM (#42867393)

No, the full R1 model is ~650GB. There are quantized version that quantize it down to ~150GB.

What you can run locally are the distilled models, that is actually LLama and Qwen weights further trained on R1's output

[ Parent | on HN ]

Re:DeepSeek proves the future of LLMs is open-source by deepsquirrelnet on Wednesday January 29, 2025 @11:16AM (#42866867)

At least a TB of VRAM to load it in fp16. They distilled to smaller models, which do not perform as well, but can be run on a single GPU. Full R1 is big though.

[ Parent | on HN ]

Re:DeepSeek proves the future of LLMs is open-source by nickthegreek on Thursday January 30, 2025 @11:14AM (#42879157)

fp16? I thought it was trained at fp8.

[ Parent | on HN ]