With Opus 5.5, Max would be around 4.5x more expensive than Medium reasoning.
That MAI Code 2 is within 4% of the performance at Max seems unlikely, to say the least.
Actual performance and a price comparison to Opus 5.5 at Medium (Anthropic's recommended setting) would be interesting.
Can someone help me understand how they arrive at this figure?
In all seriousness I have no idea how they came to that conclusion. We really need better standards here for evaluating these things but obviously that is difficult.
98 can be good I.e. human test score or funds recovered from debt.
I think for AI tasks it is somewhere between but 2% losses will add up (or you need more loops to correct and hence cost)
Meanwhile, Deepseek 4.1 flash is 3000% better at 3.5% of the cost, in that it gets work done (I have never had anything really done with GPT 6.1 Sol).
This is real token usage in a complicated monorepo codebase for adding a feature:
---
315,207,850 tok
Cache hit
99.6%
Uncached input
1,187,911 tok
Cached input
313,640,960 tok
Output
378,979 tok
---
It ended up creating a few PRs, with a total of around 6000 lines of code and the feature works as expected. Total cost was around $1.35. GPT 6.1 Sol would have cost around $40 for this and from my experience with it, it would most definitely have had so much bugs that it would not be worth fixing.
So, Microsoft would charge you $16 to waste your time even more. What are we even doing?
"MAI Code 2 reaches 98% of the coding quality of GPT-6.1 Sol at about 40% of the price. And against Opus 5.5, it reaches 96% performance for just 5% of the price."