News for hackers. Stuff that matters.
Re:OpenAI Misled You on RLHF by markisus on Sunday August 17, 2025 @12:55PM (#44933005)
Just speculating but proximity to a reference answer is a much denser reward signal. In contrast, parsing out a final answer into a pass/fail only provides a sparse reward signal.
Re:OpenAI Misled You on RLHF by krackers on Thursday August 21, 2025 @02:05AM (#44969535)
Yup, RLVR as implemented by Deepseek et al. use only outcome supervision instead of process supervision. There have been attempts to do process supervision though.