News for hackers. Stuff that matters.
Re:Microsoft AI: Seven new world-class models at scalable pr... by tyingq on Wednesday November 18, 2026 @12:29PM (#50549241)
This seems a pretty bold statement, especially the Opus 5.5 comparison.

"MAI Code 2 reaches 98% of the coding quality of GPT-6.1 Sol at about 40% of the price. And against Opus 5.5, it reaches 96% performance for just 5% of the price."

Re:Microsoft AI: Seven new world-class models at scalable pr... by user43928 on Wednesday November 18, 2026 @12:50PM (#50549559)
5% of the price compared to Opus 5.5 Max reasoning, and that's at a 50% promotional discount for MAI Code 2, I believe.

With Opus 5.5, Max would be around 4.5x more expensive than Medium reasoning.

That MAI Code 2 is within 4% of the performance at Max seems unlikely, to say the least.

Actual performance and a price comparison to Opus 5.5 at Medium (Anthropic's recommended setting) would be interesting.

Re:Microsoft AI: Seven new world-class models at scalable pr... by spprashant on Wednesday November 18, 2026 @02:10PM (#50550664)
> 98% of the coding quality

Can someone help me understand how they arrive at this figure?

Re:Microsoft AI: Seven new world-class models at scalable pr... by Melatonic on Wednesday November 18, 2026 @04:01PM (#50552068)
Dont worry - it only hallucinates security breaking bugs in 2% of the code it produces. Comparatively :-D

In all seriousness I have no idea how they came to that conclusion. We really need better standards here for evaluating these things but obviously that is difficult.

Re:Microsoft AI: Seven new world-class models at scalable pr... by vibebingo on Wednesday November 18, 2026 @04:16PM (#50552233)
98 can be bad. E.g. uptime or aviation safety.

98 can be good I.e. human test score or funds recovered from debt.

I think for AI tasks it is somewhere between but 2% losses will add up (or you need more loops to correct and hence cost)

Re:Microsoft AI: Seven new world-class models at scalable pr... by WASDx on Wednesday November 18, 2026 @04:33PM (#50552411)
I think they mean "98% of the average score on these old benchmarks which are no longer representative".
Re:Microsoft AI: Seven new world-class models at scalable pr... by tomrod on Wednesday November 18, 2026 @04:27PM (#50552352)
How much % of price to reach 200% of performance?
Re:Microsoft AI: Seven new world-class models at scalable pr... by Rodmine on Wednesday November 18, 2026 @04:51PM (#50552692)
>98% of the coding quality of GPT-6.1

Meanwhile, Deepseek 4.1 flash is 3000% better at 3.5% of the cost, in that it gets work done (I have never had anything really done with GPT 6.1 Sol).

This is real token usage in a complicated monorepo codebase for adding a feature:

---

315,207,850 tok

Cache hit

99.6%

Uncached input

1,187,911 tok

Cached input

313,640,960 tok

Output

378,979 tok

---

It ended up creating a few PRs, with a total of around 6000 lines of code and the feature works as expected. Total cost was around $1.35. GPT 6.1 Sol would have cost around $40 for this and from my experience with it, it would most definitely have had so much bugs that it would not be worth fixing.

So, Microsoft would charge you $16 to waste your time even more. What are we even doing?

Re:Microsoft AI: Seven new world-class models at scalable pr... by ta93754829 on Wednesday November 18, 2026 @06:12PM (#50553687)
that's the thing with coding quality. as we all know, one line of code can ruin everything, so every small increment matters much more than the cost.