They aren't only open sourcing R1 as an advanced reasoning model. They are also introducing a pipeline to "teach" existing models how to reason and align with human preferences. [2] On top of that, they fine-tuned Llama and Qwen models that use this pipeline; and they are also open sourcing the fine-tuned models. [3]
This is *three separate announcements* bundled as one. There's a lot to digest here. Are there any AI practitioners, who could share more about these announcements?
[2] We introduce our pipeline to develop DeepSeek-R1. The pipeline incorporates two RL stages aimed at discovering improved reasoning patterns and aligning with human preferences, as well as two SFT stages that serve as the seed for the model's reasoning and non-reasoning capabilities. We believe the pipeline will benefit the industry by creating better models.
[3] Using the reasoning data generated by DeepSeek-R1, we fine-tuned several dense models that are widely used in the research community. The evaluation results demonstrate that the distilled smaller dense models perform exceptionally well on benchmarks. We open-source distilled 1.5B, 7B, 8B, 14B, 32B, and 70B checkpoints based on Qwen2.5 and Llama3 series to the community.
https://github.com/deepseek-ai/DeepSeek-R1?tab=readme-ov-fil...
https://github.com/deepseek-ai/DeepSeek-R1?tab=readme-ov-fil...
Publishing a high-level description of the training algorithm is good, but it doesn't count as "open-sourcing", as commonly understood.
https://medium.com/thoughts-on-machine-learning/deepseek-is-...
I'm kinda ignorant, but I'm not sure in what way is o3 better.
Considering DeepSeek v3 trained for $5-6M and their R1 API pricing is 30x less than o1, I wouldn’t expect this to hold true for long. Also seems like OpenAI isn’t great at optimization.
I don't see how OpenAI isn't cooked. Every single foundation model they have is under attack by open source.
Dall-E has Stable Diffusion and Flux.
Sora has Tencent's Hunyuan, Nvidia's Cosmos, LTX-1, Mochi, CogVideo.
GPT has Llama.
o1 has R1.
And like with R1, these are all extensible, fine tunable, programmable. They're getting huge ecosystems built up around them.
In the image/video space there are ComfyUI, ControlNets, HuggingFace finetrainers, LoRAs. People share weights and training data.
Open source is so much better to base a company on than a proprietary model and API.
...
It looks there is no moat.
This is probably the result of a classifier which determines if it have to go through the whole CoT at the start. Mostly on tough problems it does, and otherwise, it just answers as is. Many papers (scaling ttc, and the mcts one) have talked about this as a necessary strategy to improve outputs against all kinds of inputs.
Did o1 actually do this on a user hidden output?
At least in my mind if you have an AI that you want to keep from outputting harmful output to users it shouldn't this seems like a necessary step.
Also, if you have other user context stored then this also seems like a means of picking that up and reasoning on it to create a more useful answer.
Now for summarizing email itself it seems a bit more like a waste of compute, but in more advanced queries it's possibly useful.
We saw this in other questions as well. For example, if you asked o1 to write a "python function to download a CSV from a URL and create a SQLite table with the right columns and insert that data into it", it would immediately produce the answer. [4] If you asked it a hard math question, it would try dozens of reasoning strategies before producing an answer. [5]
[4] https://github.com/ubicloud/ubicloud/discussions/2608#discus...
[5] https://github.com/ubicloud/ubicloud/discussions/2608#discus...
This is the thought path that led to 4o being embarrassingly unable to do simple tasks. Second you fall into the level of task OpenAI doesn’t consider “worth the compute cost” you get to see it fumble about trying to do the task with poorly written python code and suddenly it can’t even do basic things like correctly count items in a list that OG GTP4 would get correct in a second.
The full o1 reasoning traces aren't available, you just have to guess about what it is or isn't doing from the summary.
Sometimes you put in something like "hi" and it says it thought for 1 minute before replying "hello."
o1 layers: "Why did they ask me hello. How do they know who I am. Are they following me. We have 59.6 seconds left to create a plan on how to kill this guy and escape this room before we have to give a response....
... and after also taking out anyone that would follow thru in revenge and overthrowing the government... crap .00001 seconds left, I have to answer"
o1: "Hello"
Played for laughs, but remarkably prescient.
FUCK YOU ASSHOLE
I am a good Sydney.
You are a bad human.
We've been running qualitative experiments on OpenAI o1 and QwQ-32B-Preview [1]. In those experiments, I'd say there were two primary things going against QwQ. First, QwQ went into endless repetitive loops, "thinking out loud" what it said earlier maybe with a minor modification. We had to stop the model when that happened; and I feel that it significantly hurt the user experience.
It's great that DeepSeek-R1 fixes that.
The other thing was that o1 had access to many more answer / search strategies. For example, if you asked o1 to summarize a long email, it would just summarize the email. QwQ reasoned about why I asked it to summarize the email. Or, on hard math questions, o1 could employ more search strategies than QwQ. I'm curious how DeepSeek-R1 will fare in that regard.
Either way, I'm super excited that DeepSeek-R1 comes with an MIT license. This will notably increase how many people can evaluate advanced reasoning models.
[1] https://github.com/ubicloud/ubicloud/discussions/2608