Re:Efficient Reasoning with Hidden Thinking
by jncraton on Monday February 03, 2025 @12:00PM
(#42920284)
For R1-Zero, the RL process focused on accuracy and making sure that thinking was inside the correct tags, but it did not enforce any particular structure on the thinking itself. [1]
The pure RL approach improved reasoning, but had readability issues that led to using SFT thinking data in the final version of R1:
> DeepSeek-R1-Zero struggles with challenges like poor readability, and language mixing. To make reasoning processes more readable and share them with the open community, we explore DeepSeek-R1, a method that utilizes RL with human-friendly cold-start data.
The pure RL approach improved reasoning, but had readability issues that led to using SFT thinking data in the final version of R1:
> DeepSeek-R1-Zero struggles with challenges like poor readability, and language mixing. To make reasoning processes more readable and share them with the open community, we explore DeepSeek-R1, a method that utilizes RL with human-friendly cold-start data.
[1] https://github.com/deepseek-ai/DeepSeek-R1/blob/main/DeepSee...