LLM conversations online are so weird. Whenever I read things like this it’s like I’m living in a different world than the other person.
GPT4 was almost useless compared to what we have available today.
Opus 4.6 and 4.8 are basically indistinguishable from Fable and Sonnet 5. 4.7 was a hot mess. The guardrails on 4.8 and 5.0 make them worse than 4.6 for many tasks. So, even if Fable is theoretically better, refusals/downgrades make it a worse product in practice. Who cares if it outperforms on 1-2% of real world tasks if 5-10% of tasks are blocked?
I’d bet most people could be downgraded to a 12 month old frontier model, and not notice for a week or so.
Anthropic’s big problem is that open weight models are 0-6 months behind. So, their product is commoditized and margins are never going to be good.
This is another unbelievable claim. I actually used frontier models from 12 months ago and they were completely different.
You'd think if there had been that many dramatic improvements I'd have to babysit an LLM less frequently.
Yes.
>So far there is no signs it is the case
How could you possibly know this?
There's a lot of "We fired a lot of people because we're sheep and now we're having to hire some of them back" stories. And a lot of "A few engineers are doing a lot more, but we're not quite sure how to turn that into actual money" stories. And even more "We told everyone to tokenmaxx, and they did, and then we realised it was costing too much, so we stopped," stories.
But there really hasn't been a deluge of "AI has cut costs and increased profits while also improving quality" stories.
There has been a small outbreak of vibe-startups offering fairly generic services - mostly marketing and adjacent - who are doing okay, possibly.
But established tech? Doubt.
The point is, is anyone getting any value from it?
No, you're right, no one is getting any value from it.
Sure, things got better. But I'd call it iterative more than revolutionary. I still wouldn't trust any of the models to do anything meaningful unattended. They all still do dumb shit all the time.
Plus, even if they were genuinely dramatically better, the businesses sure as hell aren't. They're burning money left and right, they have no moat, Chinese open models are basically equivalent these days. What's the path to profitability, or hell, break-even? How do you envision this being anything but a giant money pit?