News for hackers. Stuff that matters.

A comment on OpenAI Reinforcement Fine-Tuning Research Program. Parent comment

Re:OpenAI Reinforcement Fine-Tuning Research Program by changoplatanero on Friday December 06, 2024 @04:17PM (#42344547)
Note that this reinforcement finetuning is something different than regular RLHF/DPO post training
[ Parent | on HN ]
Re:OpenAI Reinforcement Fine-Tuning Research Program by whimsicalism on Friday December 06, 2024 @05:11PM (#42345095)
Is it? We have no idea.
[ Parent | on HN ]
Re:OpenAI Reinforcement Fine-Tuning Research Program by changoplatanero on Friday December 06, 2024 @10:06PM (#42346917)
Yes it is. In RLHF and DPO you are optimizing the model output for human preferences. In the reinforcement fine tuning that was announced today you are optimizing the hidden chain of thought to arrive at a correct answer, as judged by a predefined grader.
[ Parent | on HN ]
Re:OpenAI Reinforcement Fine-Tuning Research Program by whimsicalism on Saturday December 07, 2024 @10:16AM (#42350299)
I mean i think it could easily be PPO post training. if your point is that the rewards are different, sure
[ Parent | on HN ]