There was some chatter on Twitter, Reddit and here when an earlier version of this came out where people normalized it by completed AA or Epoch task and Claude models still came out well ahead.
In practice OpenAI subscriptions are more competitive than they appear because OpenAI does a lot more subscription resets. But that's hard to account for!
Per task, Opus 5.5 costs ~6x of Sol 6.1. Then you add on the lack of a 5-hour limit, no limit on which models (like Fable's 50% of usage limit) you can use, very frequent resets, and that OpenAI doubled the usage limits after that analysis was performed.
Even though the Claude subscription is equivalent to 5x the $ value of API priced tokens, it seems like Codex is the better value for getting things done.
https://artificialanalysis.ai/?models=claude-fable-5-2-high%...
I don't think so? I think OpenAI had planned to half the limits but decided not to go ahead with it, but this analysis was done at the limits we have now. I could be wrong about this though?
> Opus 5.5 costs ~6x of Sol 6.1.
My point in part is that you can't compare like this.
Opus 5.5 is a more capable coding model than Sol 6.1. AA benchmarks show Opus 5.5 max effort scoring 62 vs Sol 6.1 max effort on 54: https://artificialanalysis.ai/agents/coding-agents?agents=de...
Unfortunately AA doesn't report non-Max efforts for their coding benchmarks.
But no one runs them on max. For Opus the advice is to usually run it on medium effort, and I've found that advice to be good.
For me to get equivalent results on Sol I need to run it on high effort, and I need to use Astra as an orchestrator (whereas Opus medium does great as an orchestrator too).
That combination pushes the blended token pricing up a lot.
I can't give you hard numbers on this, but at the moment my Codex credits get used up a lot quicker than my Claude credits on similar tasks in the same codebase.
I'm glad we have two great models competing, but at the moment I'd say Opus has the edge.
I think it is a narrow view to only consider the AA coding agent index as the only metric, especially with the models running at max.
For the link you shared, Opus used 3.7x as many tokens, and 2.5x the time to get its score. In the real-world building things with these tools, that kind of increase in time and cost (or subscription usage) matters, at least to me.
When comparing at more reasonable medium or high levels of reasoning, the gap between Opus and Sol narrows considerably. Opus being about 2 higher than Sol at the same reasoning level. However, Sol 6.1 xhigh matches Opus 5.5 high on scores (52), but while still being slightly faster and costing 78% less ($0.76 versus $3.46).
https://artificialanalysis.ai/models?models=claude-opus-5-5-...
> https://artificialanalysis.ai/models?models=claude-opus-5-5-...
Yes, but my point is that these are non-coding benchmarks at these non-max levels.
On coding at non-max Opus really does seem noticeably better, and my subscription seems to go longer. Look at the other comments in this thread - every single one says the same thing.
It seems like most analysis is basically "how many tokens do you get with a subscription and how much would those tokens cost at API rates". Looking at it that way, you do get more dollars of API equivalent usage for your subscription.
But Anthropic's API pricing is a lot higher than OpenAI's (Opus 5.5 is 2x of GPT-6.1 Sol). The number of tokens consumed per completed task is also far higher than GPT-6. So when you look at the number of tasks you can complete within a subscription, OpenAI looks like a much better value.
That is before you consider that OpenAI subscriptions (other than the cheapest one) don't have 5-hour usage limits, and also generally give you multiple free resets per month.