Re:MaxxBench – "Nice DeepSWE score. But is it true?"
by lubitelpospat on Saturday November 07, 2026 @09:03PM
(#50409858)
I’ve been noticing that although Astra performs better on tasks with well-defined tasks with clearly outlined definition of done(Claudis/Neuralese is invading our language indeed!), it doesn’t follow instructions well and often ignores instructions defined in text which is something you’d expect from an LLM. On the other side, GPT 6.1 Sol, while performing worse than Astra on benchmarks, is way better at following instructions and in vibe-tests on certain domains. I suspect Astra is more focused on “agentic” tasks, and is not entirely supposed to pay too much attention to the instructions of a meat bag?
[ Parent
| on HN ]
There are no comments on this comment yet.