News for hackers. Stuff that matters.
Re:RLHF a LLM in <50 lines of Python by nextaccountic on Sunday February 11, 2024 @11:40AM (#39336168)
> I'm not a fan of the RL/SL dichotomy, because the line gets so foggy. If you squint, every loss is a negative reward, and every policy improvement a supervised target.

Isn't this just because reinforcement learning and supervised learning are both optimization problems?

Re:RLHF a LLM in <50 lines of Python by espadrine on Sunday February 11, 2024 @11:53AM (#39336275)
In part, yes! But also because what used to define it was the human-curated datasets: SL contained input/output pairs, while RL contained episodes with sporadic rewards.

Nowadays, many datasets have different forms or are synthetic. DPO uses datasets with both positive and negative examples (instead of just a target output as with traditional SL); RLHF uses synthetic rewards.