Show HN: RewardHackBench: Using sandboxes to stop agents from cheating

Posted by rotemtam on Wednesday June 17, 2026 @07:40AM
from the show-hn dept.

https://github.com/islo-labs/reward-hack-bench

hey all,

happy to share research i've been working on for islo.dev in recent months.

ever since the cheating agents (https://debugml.github.io/cheating-agents/) paper came out, revealing reward hacking was 4x more prevalent than previously estimated, i've been looking into how we can deal with the issue

the common approach (taken by the tbench team) is post hoc trajectory analysis.

i've been interested in the idea of reframing the problem as an endpoint security problem and tackling it via sandbox

i hope you find it interesting, and thanks to the islo.dev team for sponsoring this

happy to answer any Qs

( 3 comments | 6 points | on HN )

Re:Show HN: RewardHackBench: Using sandboxes to stop agents ... by yonSpektor on Wednesday June 17, 2026 @08:14AM (#48569302)

Curious what the distribution of hacking strategies looked like across different models — would expect RL-heavy vs RLHF models to cheat very differently.

[ Parent | on HN ]

Re:Show HN: RewardHackBench: Using sandboxes to stop agents ... by adamgold7 on Wednesday June 17, 2026 @07:56AM (#48569115)

love this. we are actually looking at reward hacking from a cyber security perspective - refreshing (unless you're from Israel).

Any collaborators that want to join us?

[ Parent | on HN ]

Re:Show HN: RewardHackBench: Using sandboxes to stop agents ... by matankleyman1 on Wednesday June 17, 2026 @10:15AM (#48570885)

that's one of the biggest long term issues with agents that no one has real interest talking about.

[ Parent | on HN ]