News for hackers. Stuff that matters.
Re:OpenAI Announces New SWE-Lancer Benchmark by zone411 on Tuesday February 18, 2025 @02:15PM (#43093836)
Re:OpenAI Announces New SWE-Lancer Benchmark by dang on Tuesday February 18, 2025 @05:58PM (#43096215)
Comments moved thither. Thanks!
Re:OpenAI Announces New SWE-Lancer Benchmark by bufferoverflow on Tuesday February 18, 2025 @01:11PM (#43093083)
And how do you evaluate if the task was completed correctly? There are nearly infinite ways to solve a given software dev problem, if the problem isn't trivial (and I hope they are not benchmarking trivial problems).
Re:OpenAI Announces New SWE-Lancer Benchmark by riku_iki on Tuesday February 18, 2025 @04:36PM (#43095410)
paper says they created e2e tests to check if task completed successfully.