Re:How We Broke Top AI Agent Benchmarks: And What Comes Next
by BugsJustFindMe on Saturday April 11, 2026 @07:37PM
(#47734910)
Intel basically benchmaxxed their compiler optimizations. They used detailed knowledge of the benchmark to make their compiler generate machine code to do better on the benchmark in a way that was not beneficial for non-benchmark scenarios.
[ Parent
| on HN ]
Re:How We Broke Top AI Agent Benchmarks: And What Comes Next
by bee_rider on Sunday April 12, 2026 @02:07AM
(#47736600)
I assumed as much, I’m just wondering what exactly they did. For example IIRC some phone company would detect that a benchmark was running by checking for the program name, and then allow the clock to boost higher (increase thermal limits) if it was a benchmark (like you could literally avoid the cheating behavior by changing the name of the program being run).