Re:MaxxBench – "Nice DeepSWE score. But is it true?"
by mydreamof on Saturday November 07, 2026 @03:14PM
(#50407626)
One thing to consider is that even when some model has big gap between those two benchmark it doesn't mean it is much worse. It could be worse becasue he has assuptions based on training but on real scenario it wouldn't have it so it still could achive great results
[ Parent
| on HN ]
Re:MaxxBench – "Nice DeepSWE score. But is it true?"
by kqr on Saturday November 07, 2026 @08:23PM
(#50409683)
This test still says something about how well the model adapts to subverted expectations. Some models get more stuck in local regions of their latent spaces than others.
But I agree witih you an important piece of the generalisation puzzle is missing.