But the odd thing is that even the previous generations of these models aren't on Artificial Analysis. It's been months since version 1 of some of these models was announced, but none of these are on AA. Are they only providing them with anti-benchmarking language or something?
John here from the team that built Microsoft Decision 1 and 1.1.
Microsoft Decision 1.1 is on the Artificial Analysis Decision Index v2.1 leaderboard, where score 68, above Gemini Decision 1, GPT-6 Luna Decisions, and Jev 1.14. [1]
We also compare ourselves to other decision models in our blog post on Microsoft's Command Line blog from November 17 [2]. We name all of the public benchmarks we evaluated against in the post, and provide a breakdown by category.
[1] https://artificialanalysis.ai/evaluations/artificial-analysi... [2] https://commandline.microsoft.com/microsoft-decision-model-g...