Every story and comment from the Hacker News API, in a layout borrowed from 2001.
A comment on How We Broke Top AI Agent Benchmarks: And What Comes Next. Parent comment
Labs need accurate benchmark measurements, at least internally, to figure out what model improvements actually matter.
Having models exploit benchmarks serves no purpose. If they wanted to make their models look better than they are, they could just make the data up.
There are no comments on this comment yet.
Labs need accurate benchmark measurements, at least internally, to figure out what model improvements actually matter.
Having models exploit benchmarks serves no purpose. If they wanted to make their models look better than they are, they could just make the data up.