News for hackers. Stuff that matters.

A comment on Why SWE-bench Verified no longer measures frontier coding capabilities. Parent comment

Re:Why SWE-bench Verified no longer measures frontier coding... by Schlagbohrer on Monday April 27, 2026 @04:53AM (#47919220)
Was that using an RNG? Or is the entire game deterministic?
[ Parent | on HN ]
Re:Why SWE-bench Verified no longer measures frontier coding... by CamperBob2 on Monday April 27, 2026 @11:40AM (#47923059)
It used an RNG. The usual practice back then was to spin a counter while waiting for keypresses, so that might affect the question when dealing with an external harness, I suppose.
[ Parent | on HN ]