News for hackers. Stuff that matters.

A comment on Show HN: I RL-trained an agent that trains models with RL (for –$1.3k).

Re:Show HN: I RL-trained an agent that trains models with RL... by azertyvode on Wednesday July 15, 2026 @04:53AM (#48918040)
How do you prevent the agent from reward-hacking the hidden eval? e.g. writing training data that effectively leaks the eval distribution rather than teaching a general skill?
[ Parent | on HN ]
Re:Show HN: I RL-trained an agent that trains models with RL... by Danau5tin on Wednesday July 15, 2026 @04:58AM (#48918074)
Great question. Today the answer is that there is nothing stopping this, but you are right this is a 100% valid reward hacking possibility.

No signs of it in the run I did, but for any future larger scale runs, it would be wise to dedicate some focus to prevent this.

[ Parent | on HN ]