DeepSeek-R1 with Dynamic 1.58-bit Quantization

Re:DeepSeek-R1 with Dynamic 1.58-bit Quantization by generalizations on Tuesday January 28, 2025 @08:55AM (#42852301)

In my experience with deepseek and o1, openai's big talk about (and investment into) hallucination avoidance might save their hides here. Deepseek may be smarter, and understand complex problems better, but it also seems to make mistakes more often. (It's as if it's comprehension is better, but it's worse at memorization/recall.)

Need an LLM to one-shot some complex network scripting? as of last night, o1 is still where its at.

[ Parent | on HN ]

Re:DeepSeek-R1 with Dynamic 1.58-bit Quantization by flir on Tuesday January 28, 2025 @09:26AM (#42852629)

My experience gels with yours. Given the same code sample, DeepSeek has better, more creative suggestions about how to improve it, but it can't implement them without breaking the code. o1, generally, can implement DeepSeek's suggestions successfully. I think chaining them together might have quite interesting results.

[ Parent | on HN ]

Re:DeepSeek-R1 with Dynamic 1.58-bit Quantization by wordpad25 on Tuesday January 28, 2025 @10:13AM (#42853216)

Is there a tool that can automate chaining like that?

[ Parent | on HN ]

Re:DeepSeek-R1 with Dynamic 1.58-bit Quantization by throwup238 on Tuesday January 28, 2025 @10:18AM (#42853297)

Aider has an architect mode where it asks one model to plan out the changes and another to actually write the code.

[ Parent | on HN ]

Re:DeepSeek-R1 with Dynamic 1.58-bit Quantization by manmal on Tuesday January 28, 2025 @05:33PM (#42858989)

I've used it today, with R1 as architect and Sonnet as editor model. So far, this works great. There's no need to use a reasoning model as editor IMO.

Alex (https://alexcodes.app) also does this now btw.

[ Parent | on HN ]

Re:DeepSeek-R1 with Dynamic 1.58-bit Quantization by HarHarVeryFunny on Tuesday January 28, 2025 @10:30AM (#42853441)

That's ok if all you want to know is which model should I use today, but a test like that is totally dependent on training data, and there is no reason to expect that either DeepSeek-V3 (the base model for R1) or the additional training data for R1 is that same as what OpenAI used for O1 and whatever base model it was built on.

The benchmark comparisons are perhaps, for now, the best way to compare reasoning prowess of R1 vs O1, since it seems pretty certain they both trained for those cases.

I think the real significance of R1 isn't the released model/weights itself, but more the paper detailing (sans training data) how to replicate it, and how effective "distillation" (i.e. generate synthetic reasoning data for SFT) can be to enhance reasoning even without using RL.

[ Parent | on HN ]