News for hackers. Stuff that matters.

A comment on DeepSWE: A contamination-free benchmark for long-horizon coding agents.

Re:DeepSWE: A contamination-free benchmark for long-horizon ... by dnnssl2 on Tuesday May 26, 2026 @05:44PM (#48286403)
70% at launch seems pretty saturated, why ship a benchmark frontier models are about to top out on?
[ Parent | on HN ]
Re:DeepSWE: A contamination-free benchmark for long-horizon ... by vanuatu on Tuesday May 26, 2026 @08:29PM (#48287900)
sell data for them to hillclimb :)
[ Parent | on HN ]