Re:MaxxBench – "Nice DeepSWE score. But is it true?"
by kgeist on Sunday November 08, 2026 @12:05AM
(#50410546)
DS4's architecture was designed to squeeze as much context as possible into available VRAM, to save on GPUs and serve as many users as possible. As a result, DS4 often loses track of the original prompt (or what it did five minutes ago), because during token generation it doesn't see the context in full; it sees an overcompressed version that loses a lot of detail. Other models use lossy compression too, but from what I know, DS4 compresses everything more aggressively.
And DS4.1 now seems to compress things even more aggressively than DS4 (4x)
Every once in a while I come across comments that just hates DS. I wonder if it's a matter of how people use it or the prompt. I admit pre-v4.1 was pretty bad, hallucinating too much and not getting the abstractions right so I had to correct it often.
I also have $200 Claude subscription (now called "Ultra"). Fable gets abstractions right on highly complex problems (e.g. defining protocol that spans multiple services) that DS simply cannot do. However, DS is comparable to Opus 5.5, although less "intelligent", but DS excels at speed which is what I prefer than waiting minutes on Opus 5.5.