News for hackers. Stuff that matters.
Re:RLHF a LLM in <50 lines of Python by ilaksh on Sunday February 11, 2024 @04:34PM (#39338972)
How do you normally do DPO? Is that built in to PyTorch or something?

Theoretically the hard part is collecting the examples with rejections etc.

Re:RLHF a LLM in <50 lines of Python by patelajay285 on Sunday February 11, 2024 @04:42PM (#39339053)
Collecting data is hard, but the library is also a synthetic data generation library, so for example you can create the data for DPO fully synthetically, check out the self-rewarding LLMs example: https://datadreamer.dev/docs/latest/pages/get_started/quick_...