News for hackers. Stuff that matters.

A comment on DeepSeek-R1 with Dynamic 1.58-bit Quantization. Parent comment

Re:DeepSeek-R1 with Dynamic 1.58-bit Quantization by brookst on Tuesday January 28, 2025 @09:52AM (#42852954)
Or if you want a large model but don’t need high performance, get a Mac with 128GB UMA.
[ Parent | on HN ]
Re:DeepSeek-R1 with Dynamic 1.58-bit Quantization by hdjjhhvvhga on Tuesday January 28, 2025 @05:48PM (#42859155)
How many tokens/s would you get in such a setup?
[ Parent | on HN ]
Re:DeepSeek-R1 with Dynamic 1.58-bit Quantization by brookst on Wednesday January 29, 2025 @09:32AM (#42865294)
This Reddit thread says an m3 max 128GB gets 23 tokens/sec with deepseek r1 32B, and 4 tokens / sec with 70b: https://www.reddit.com/r/LocalLLaMA/comments/1i69dhz/deepsee...
[ Parent | on HN ]