News for hackers. Stuff that matters.
Re:Using GRPO to Beat o1, o3-mini and R1 at "Temporal Clue" by a_wild_dandan on Thursday March 06, 2025 @03:55PM (#43284988)
"We suspect the models may have converged too early on greedy strategies that worked out of the gate, but potentially limited their long-term prospects. A logical next step would explore approaches that encourage diverse responses, or that build capabilities incrementally (like with curriculum learning), or that assign larger rewards to particularly outstanding solutions incentivizing thorough exploration. Additionally, we noted interesting patterns in output length during training. Initially, responses grew longer, then stabilized, before diverging near the end of training, with the 14B model’s responses getting longer and the 32B model’s response lengths collapsing, especially after reaching peak performance."

There are no comments on this comment yet.