Re:DeepSeek-R1 with Dynamic 1.58-bit Quantization
by ant6n on Wednesday January 29, 2025 @05:44AM
(#42863556)
Have you tried running the full R1 model with that? People in sibling comments mention high end EPYCs gor a 10K machine, but I’m curious whether it’s possible to make a 1-2K machine that could still run those big models simply because they fit in RAM.
[ Parent
| on HN ]
Re:DeepSeek-R1 with Dynamic 1.58-bit Quantization
by immibis on Wednesday January 29, 2025 @07:23AM
(#42864083)
I spent about $3000 on my machine, have the cheapest Threadripper CPU and 256GB of RAM, so no, 600GB won't fit in RAM on a $2K machine.
But everyone is using the distilled models which are much smaller.