News for hackers. Stuff that matters.

A comment on Supervised fine tuning on curated data is reinforcement learning.

Re:Supervised fine tuning on curated data is reinforcement l... by markisus on Tuesday July 29, 2025 @10:51PM (#44730533)
Something seems off with equation (5).

Just imagining Monte Carlo sampling it, the middle expectation will have a bunch of zeros due to the indicator function and the right expectation won’t.

I can make the middle expectation be as close to zero as I like by making the success threshold sufficiently high.

[ Parent | on HN ]
Re:Supervised fine tuning on curated data is reinforcement l... by chongliqin on Wednesday July 30, 2025 @12:20PM (#44736219)
Ah yes you are right the rhs was meant to be proportional to the middle expectation (see the equation below), for equality the rhs needs to be multiplied by a normalization constant independent of theta. Note this doesn't affect the bounds as the constant is the same across equations. Will update the paper to incorporate.
[ Parent | on HN ]