News for hackers. Stuff that matters.
Re:MaxxBench – "Nice DeepSWE score. But is it true?" by saejox on Saturday November 07, 2026 @01:34PM (#50406817)
There is always a huge discrepancy between benchmarks and real world capabilities.

For my use case deepseek 4.1 flash is almost as dumb as haiku 5.5. Both are fast garbage producers.

Re:MaxxBench – "Nice DeepSWE score. But is it true?" by jeffbee on Saturday November 07, 2026 @01:40PM (#50406852)
I was beginning to worry I had overlooked the inner beauty of DeepSeek. When I have tried it, it was totally useless. Never understood HN cheering for it.
Re:MaxxBench – "Nice DeepSWE score. But is it true?" by AlotOfReading on Saturday November 07, 2026 @02:10PM (#50407119)
Don't know what you're doing, but it's good enough for the reasonably sized tasks I give LLMs. It's also a much better price. My browser stopped rendering the UI frame recently because I messed up permissions on /usr/share/icons. Took DS 4.1 flash about 3 minutes and a few cents to diagnose it. Same harness with opus 5 took twice as long and several dollars.

Moreover, the months I've used anthropic/OAI exclusively my API token costs are in the hundreds (or sometimes thousands). With DS flash as the first agent I reach for, I rarely exceed $40/mo.

Re:MaxxBench – "Nice DeepSWE score. But is it true?" by jeffbee on Saturday November 07, 2026 @02:24PM (#50407217)
I meant for generative coding. I'm sure it can do a few things here and there.
Re:MaxxBench – "Nice DeepSWE score. But is it true?" by andreypopp on Saturday November 07, 2026 @02:39PM (#50407364)
How did you use it? I was using it as "implementor" subagents orchestrated by opus 5.5 and it was pretty good and fast.
Re:MaxxBench – "Nice DeepSWE score. But is it true?" by AlotOfReading on Saturday November 07, 2026 @02:50PM (#50407450)
I figured it was assumed I was including that? It's been fine as long as I break tasks up into reasonably sized chunks, which is fine because I think you should only be generating as much as you can comfortably review at a time.
Re:MaxxBench – "Nice DeepSWE score. But is it true?" by jameson on Saturday November 07, 2026 @06:53PM (#50409178)
I'm a heavy DS v4.1 user and been satisfied with its performance. I use it just like you described -- "... break tasks up into reasonably sized chunks...".

Every once in a while I come across comments that just hates DS. I wonder if it's a matter of how people use it or the prompt. I admit pre-v4.1 was pretty bad, hallucinating too much and not getting the abstractions right so I had to correct it often.

I also have $200 Claude subscription (now called "Ultra"). Fable gets abstractions right on highly complex problems (e.g. defining protocol that spans multiple services) that DS simply cannot do. However, DS is comparable to Opus 5.5, although less "intelligent", but DS excels at speed which is what I prefer than waiting minutes on Opus 5.5.

Re:MaxxBench – "Nice DeepSWE score. But is it true?" by kgeist on Sunday November 08, 2026 @12:05AM (#50410546)
DS4's architecture was designed to squeeze as much context as possible into available VRAM, to save on GPUs and serve as many users as possible. As a result, DS4 often loses track of the original prompt (or what it did five minutes ago), because during token generation it doesn't see the context in full; it sees an overcompressed version that loses a lot of detail. Other models use lossy compression too, but from what I know, DS4 compresses everything more aggressively.

And DS4.1 now seems to compress things even more aggressively than DS4 (4x)

Re:MaxxBench – "Nice DeepSWE score. But is it true?" by pixelesque on Saturday November 07, 2026 @02:36PM (#50407333)
I've had significant success using DS 4 Pro and now 4.1 flash for fairly complex coding tasks on C++, Rust and Python projects at home.

I have access to Claude Code at work (and use it quite a bit), so I feel I can honestly compare them.

You do have to plan things first a bit more with DS (than say Opus can (Opus will have more chance of one-shotting something, although I generally don't think that's a good idea), although it often over-engineers and tries to be too helpful in my experience) and be a bit more explicit at times, but I feel it's quite close to Sonnet in performance for implementing plans, debugging issues and explaining how codebases work.

Re:MaxxBench – "Nice DeepSWE score. But is it true?" by mtrovo on Saturday November 07, 2026 @02:46PM (#50407426)
Hard to say if you don't share what kind of project, workflow and prompt you're trying tbh. I've been driving it mainly to write a quite big golang and android app and most of the tasks are all handled fine by DeepSeek.
Re:MaxxBench – "Nice DeepSWE score. But is it true?" by howunfortunate on Saturday November 07, 2026 @03:00PM (#50407525)
Assume about 50% of internet writing has zero tether to reality and is written out of pure motivation to shape a preferred narrative (and usually the same one).

When you see this your worldview will instantly become much clearer.

Re:MaxxBench – "Nice DeepSWE score. But is it true?" by tyzoid on Saturday November 07, 2026 @03:22PM (#50407709)
Not everyone is operating with the same constraints. As for me, once the models got good enough (~opus 4.5 levels of intelligence) that they could solve most problems that I was dealing with, I stopped following the frontier intelligence trajectory for daily use and instead followed constant intelligence and decreasing cost. Nowadays I primarily use GPT 6.1 luna, and DeepSeek 4.1 flash.

I've also been very happy playing around with qwen 3.8 27b running locally for personal projects. It sometimes can't quite get there, and that's when I'll reach for GPT.

Re:MaxxBench – "Nice DeepSWE score. But is it true?" by arjie on Saturday November 07, 2026 @03:16PM (#50407642)
This has always been a problem with the Internet. When the iPad was released, anti-Apple subreddits talked about what a garbage device it was. Wishful thinking dominates a lot of human content generators. Extracting information from humanslop is harder than avoiding it entirely.
Re:MaxxBench – "Nice DeepSWE score. But is it true?" by sjbzbeiks on Saturday November 07, 2026 @05:27PM (#50408642)
Benchmarks they share are on the much older v4 of DSF. DSF V4.1 1017 is a big step up in benchmarks and is also actually twice as many parameters.

Anyway, I use DSF v4.1 on max as much as I use Opus 5.5 on xhigh, Opus is definitely a more versatile model, but I’m not getting any value using it to implement well defined tasks that DSF v4.1 does just as well at for a fraction of the cost.

Re:MaxxBench – "Nice DeepSWE score. But is it true?" by lisplist on Sunday November 08, 2026 @12:21AM (#50410623)
I wasn’t a huge fan of DeepSeek until their very latest 4.1 flash model. The original 4.1 flash, v4 flash, v4 pro, all of it wasn’t very useful for me, but their latest release has been extremely useful.
Re:MaxxBench – "Nice DeepSWE score. But is it true?" by embedding-shape on Saturday November 07, 2026 @01:45PM (#50406896)
> There is always a huge discrepancy between benchmarks and real world capabilities.

I mean of course, the real world is messy, the models are only good for what they've been trained on, and post-training frequently takes benchmark scores into account too, leading to this sort of "Great score, sucks in the real world" syndrome that seemingly most models are affected by.

Re:MaxxBench – "Nice DeepSWE score. But is it true?" by rapind on Saturday November 07, 2026 @04:38PM (#50408295)
Different expectations maybe. In my experience, DSF 4.1 high/max is basically what Terra should be for coding, and is very acceptable (Terra is hot garbage, yes even the latest 6.1). Luna at xhigh+ stands out as a cheap implementor/reviewer of very very well defined tasks, but goes off the rails if you're not very specific, Sol and Sonnet at medium are basically powerhouses for implementation with a half decent plan, and Opus 5.5 high is an excellent planner (and implementor).
Re:MaxxBench – "Nice DeepSWE score. But is it true?" by ericpruitt on Saturday November 07, 2026 @06:48PM (#50409154)
I'm curious what quantization and version you're using. I found DeepSeek v4 to be practically useless when I was using frob/deepseek-v4-flash-0731 via Ollama, but later I switched to using unsloth/DeepSeek-V4.1-Flash-1017-GGUF, and I've been extremely happy with using it for coding tasks since then.
Re:MaxxBench – "Nice DeepSWE score. But is it true?" by efromvt on Saturday November 07, 2026 @09:34PM (#50409974)
deepseek does pretty well on my personal benchmarks, but I had a ton of problem with any openrouter providers; had to resort to always using it directly.