I think the general notion that a lot of people are using models that are overpowered for their use case is right, but I think the big question is where the line actually is and how many people fit on either side. My suspicion based on my experience is that there are significantly more people on the "actually need the frontier model" side than the bear case suggests, but I'm not speaking with any authoritative data here and could be totally wrong.

I used to work on evals/observability at Opik: https://github.com/comet-ml/opik And I can say from working with lots of users running agents at scale, there is definitely a pretty commonly reached frontier of difficulty/complexity/abstraction where the capability difference between something like Fable/Astra and smaller models is a big deal. I find this to be especially true the more autonomous your workflow gets. You can still often tackle the hardest tasks with something like Opus, it just typically involves more direct intervention and design work from the user.

But again, this is just anecdata on my part. All of this is still pretty ambiguous and lots of companies are trying to solve the problem of matching models to task difficulty (I.e. routers), with varying degrees of success.