News for hackers. Stuff that matters.
Re:Using GRPO to Beat o1, o3-mini and R1 at "Temporal Clue" by pizza on Thursday March 06, 2025 @09:06PM (#43286889)
Seems like if you want to stay in the same language, you could just add a verifiable rewards term for that w/o having to fully load up on the baggage of a base model KL penalty.
Re:Using GRPO to Beat o1, o3-mini and R1 at "Temporal Clue" by kcorbitt on Thursday March 06, 2025 @10:20PM (#43287151)
Yep. And tbh you probably don't even have to do this; the R1 paper found that just running SFT the base model with a relatively small number of monolingual reasoning traces was enough for it to get the idea and iirc they didn't even bother selecting for language specifically in the RL training looop itself.