Re:RLHF a LLM in <50 lines of Python
by patelajay285 on Sunday February 11, 2024 @04:42PM
(#39339053)
Collecting data is hard, but the library is also a synthetic data generation library, so for example you can create the data for DPO fully synthetically, check out the self-rewarding LLMs example:
https://datadreamer.dev/docs/latest/pages/get_started/quick_...
Theoretically the hard part is collecting the examples with rejections etc.