News for hackers. Stuff that matters.
Re:Why SWE-bench Verified no longer measures frontier coding... by CamperBob2 on Monday April 27, 2026 @11:44AM (#47923108)
It is not the least bit pedantic. Games were meaner back then. If you're on a time (turn)-limited section of the game, or in a vulnerable spot like the volcano, random encounters with the wizard could render the game unwinnable without dying, which would completely wreck a benchmark. Same for the thief in Zork 1. If he randomly steals your light source, you're done for. Or if the RNG dictates that you lose the fight with the troll.

Can't recall anything like that in Zork 3. (Edit: apparently you could get shot randomly when using the time machine in the Royal Museum.)

There are no comments on this comment yet.