News for hackers. Stuff that matters.

A comment on MaxxBench – "Nice DeepSWE score. But is it true?".

Re:MaxxBench – "Nice DeepSWE score. But is it true?" by YetAnotherNick on Saturday November 07, 2026 @01:52PM (#50406948)
DeepSWE was released on May and Deepseek v4 flash was released on April. By definition it couldn't be benchmaxxed on this particular benchmark. My guess is the mutation is not like real world and it is tripping small models, not benchmaxed model.
[ Parent | on HN ]
Re:MaxxBench – "Nice DeepSWE score. But is it true?" by viraptor on Saturday November 07, 2026 @02:40PM (#50407380)
It's not clear which v4 flash they used. There are 3 different releases for that one.
[ Parent | on HN ]
Re:MaxxBench – "Nice DeepSWE score. But is it true?" by YetAnotherNick on Saturday November 07, 2026 @02:54PM (#50407483)
That makes sense. I wish they run it on some model released before deepswe release like gemini-3.5-flash.
[ Parent | on HN ]
Re:MaxxBench – "Nice DeepSWE score. But is it true?" by jameson on Saturday November 07, 2026 @05:43PM (#50408736)
DS flash also has V4.1 update and supposedly achieves 77.0 on deepswe v1.1

https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash-1017

[ Parent | on HN ]
Re:MaxxBench – "Nice DeepSWE score. But is it true?" by formerly_proven on Saturday November 07, 2026 @05:52PM (#50408799)
That's a totally new model.
[ Parent | on HN ]