This just isn't true. You're talking about the textbook model of perfect competition, which is not the real world.
Commodity-like markets do tend to see outsized margins get competed away, but many markets aren't commodity-like.
Markets with network effects, economies of scale, high switching costs, significant brand differentiation, high legal and regulatory barriers to entry and heavy collusion/concentration don't function the way you claim, and there are a large number of major companies, many of them tech-related, that have delivered above-average returns year after year for many years because it's impossible for new entrants to flood the market and simply undercut them.
It's way too early to predict how this will play out for the leading AI labs like Anthropic.
The irony of your comment is that you're still using frontier models even though there are already much cheaper options available.
Can we be sure of this? The second they tried to hike the prices to be profitable, they lost a lot of clients, or a lot of clients lowered they token expenditure. In any market margins tend to go to zero. I am using frontier models, but only because they are dirty cheap if not free. If they raise the price, I just switch to a cheaper model, start exploring the option of renting a GPU-VPS or buying my own card. Or reducing my AI usage, that currently includes stupid useless things like "translate this cooking recipe from french... for free".
> But if training becomes less costly and inference margins don't compress too much, it's a totally different story.
They are being priced as the "ifs" are certainties, and not only a possibility. If anything goes a millimetre below perfect for them, their finances collapse. High margins attract competitors, and unless you have a huge moat (and their only moat is expensive training), your high margins vanish.