News for hackers. Stuff that matters.
Re:Gemini 3 for developers: New reasoning, agentic capabilities by svantana on Tuesday November 18, 2025 @11:27AM (#45968377)
SWEBench-Verified is probably benchmaxxed at this stage. Claude isn't even the top performer, that honor goes to Doubao [1].

Also, the confidence interval for a such a small dataset is about 3 percent points, so these differences could just be up to chance.

[1] https://www.swebench.com/

Re:Gemini 3 for developers: New reasoning, agentic capabilities by usaar333 on Tuesday November 18, 2025 @12:39PM (#45969455)
claude 4.5 gets 82% on their own highly customized scaffolding. (parallel compute with a scoring function). That beats Doubao