News for hackers. Stuff that matters.
Re:Why SWE-bench Verified no longer measures frontier coding... by stevex on Sunday April 26, 2026 @11:22PM (#47917383)
It's not true that there was no improvement in the rate at which models produced quality code.

Jan 2025 was Claude 3.5 Sonnet, Gemini 1.5 Pro and OpenAI had GPT-4o.

As someone who used all those models, as well as today's frontier models - today's models are a significant step up from those.

There are no comments on this comment yet.