Every story and comment from the Hacker News API, in a layout borrowed from 2001.
A comment on Why SWE-bench Verified no longer measures frontier coding capabilities. Parent comment
Both that and the SPEC compiler shenanigans are cheating by changing the test, not just over-specializing the product being benchmarked.
There are no comments on this comment yet.
Both that and the SPEC compiler shenanigans are cheating by changing the test, not just over-specializing the product being benchmarked.