what about unsuccessful tasks? those cost money too

Just throw 1000 Fable 5.2 agents at it. At least according to the marketing anyway.

It’s my understanding this model has even fewer of those than the models with higher cost per successful task.

As a result, a simple “cost per task” metric makes the most sense to use for me, when paired with the eval scores.

You're right, my metric wasn't exactly the right one. It's the intelligence/$ who matters, and Fable 5.2 is pareto optimal at all thinking levels right now.