Re:AI Agent Benchmarks Are Broken
by mycall on Friday July 11, 2025 @09:49AM
(#44532125)
SnitchBench [0] is unique benchmark which shows how aggressively models will snitch on you via email and CLI tools when they are presented with evidence of corporate wrongdoing - measuring their likelihood to "snitch" to authorities. I don't believe they were trained to do this, so it seems to be an emergent ability.
[0] https://snitchbench.t3.gg/