News for hackers. Stuff that matters.
Re:DeepSeek-R1 by tomrod on Monday January 20, 2025 @06:28PM (#42774409)
Can you recommend hardware needed to run these?
Re:DeepSeek-R1 by simonw on Monday January 20, 2025 @07:33PM (#42774892)
I'm using an M2 64GB MacBook Pro. For the Llama 8B one I would expect 16GB to be enough.

I don't have any experience running models on Windows or Linux, where your GPU VRAM becomes the most important factor.

Re:DeepSeek-R1 by rane on Tuesday January 21, 2025 @04:42AM (#42778117)
Why isn't GPU VRAM a factor on a Silicon mac?
Re:DeepSeek-R1 by Nekit1234007 on Tuesday January 21, 2025 @05:34AM (#42778523)
Because there's no VRAM. The “regular” RAM on Apple Silicon devices is shared with the GPU virtually in its entirety.
Re:DeepSeek-R1 by dragonwriter on Tuesday January 21, 2025 @11:08AM (#42781689)
On Windows or Linux you can run from RAM or split layers between RAM and VRAM; running fully on GPU is faster than either of those, but the limit on what you can run at all isn’t VRAM.
Re:DeepSeek-R1 by akhdanfadh on Wednesday January 22, 2025 @09:24AM (#42793156)
So is it possible to load the ollama deepseek-r1 70b (43gb) model on my 24gb vram + 32gb ram machine? Does this depend on how I load the model, i.e., with ollama instead of other alternatives? Afaik, ollama is basically llama.cpp wrapper.

I have tried to deploy one myself with openwebui+ollama but only for small LLM. Not sure about the bigger one, worried if that will crash my machine someway. Are there any docs? I am curious about this and how that works if any.

Re:DeepSeek-R1 by yencabulator on Tuesday January 21, 2025 @11:49AM (#42782249)
ollama runs deepseek-r1:7b on AMD 8945HS, CPU-only, at ~12 tokens/s. You can get started pretty easily in the ~7B model range, for learning purposes.