News for hackers. Stuff that matters.

A comment on How to Run DeepSeek R1 671B Locally on a $2000 EPYC Server. Parent comment

Re:How to Run DeepSeek R1 671B Locally on a $2000 EPYC Server by segmondy on Saturday February 01, 2025 @03:01PM (#42901670)
how much context size?
[ Parent | on HN ]
Re:How to Run DeepSeek R1 671B Locally on a $2000 EPYC Server by smcleod on Saturday February 01, 2025 @03:03PM (#42901686)
Just 4K. Because deepseek doesn't allow for the use of flash attention it means you can't run quantised qkv
[ Parent | on HN ]