How to Run DeepSeek R1 671B Locally on a $2000 EPYC Server

Re:How to Run DeepSeek R1 671B Locally on a $2000 EPYC Server by ranguna on Sunday February 02, 2025 @02:16AM (#42906764)

I'm not sure you can fit a decent quant of R1 in digits, 128 GB of memory is not enough for 8 and I'm not sure of 4 but I have my doubts. So you might have to go for around 1, which has a significant quality loss.

[ Parent | on HN ]

Re:How to Run DeepSeek R1 671B Locally on a $2000 EPYC Server by Cane_P on Sunday February 02, 2025 @08:28AM (#42908509)

You can connect two, and get 256 GB. But it will still not be enough to run it in native format. You will still need to use lower quant.

[ Parent | on HN ]