Don't actually spend the business effort to use them if it's not worth it, but have a business continuity plan to be able to switch over, especially if a "single source provider" going out of business or changing it's pricing could represent a existential thread to your business.
It's the same as any other tech vendor.
So at least for current gen, you're ok.
So, in most cases they have no moat. They are going to close, and while some will close due to operational cost issues as you identified, a lot more will close since they have no defensible business/revenue model.
Or, even better if they can price future API price increases in their own pricing, today.
If it's the latter, way more startups on SOTA models will become unviable once the investor subsidies dry up.
Imo. A company can use whatever wrapper services but the business they serve should be smarter.
You can actually see it in paid, web stacks. There’s free ones, esp FOSS, that are really good. So, the paid contenders try to be better at other things. There’s many of them in the market, too, despite over a decade of free alternatives.
I'm guessing that's going to be the needle that finally pops this bubble, because as neat as a lot of shit going on with various models is, nobody is going to pay what it actually costs to run the things in order to have that neatness.
You're going to have to bring some actual numbers on that speculation.
Currently their lowest-tier price is $20/month, and limits users to 80 queries per hour. If we assume that 7 cents number and our hypothetical user actually uses those 80 queries for whatever odd reason, that's $5.60 it costs OpenAI, which if that user used them for say, 8 hours (the typical work day), that's $44.80 for OpenAI on a user who's only paid $20. And that's one day, that $20 currently gets them a month of access to that 80 queries per hour. I hope for the dream of profitability that they would get throttled at some point. So the user could, in theory, already render their subscription a net loss for OpenAI with 4 workdays of steady usage.
And yes this is strictly back-of-napkin math here but it would explain why "OpenAI" and "bankrupt" keep appearing in the tech press over and over.
It's a well-established, economically rational business model to subsidize prices in order to gain market share.
Uber has been very successful at it. Amazon has had great success. It's standard.
Investors know what they're doing and know that some companies won't be able to sustain it and will go out of business. It's a calculated, known risk.
And users don't really need to be concerned. Just enjoy the low prices while they last. If the service you use goes out of business, it's pretty easy to switch to another.
> It's a well-established, economically rational business model to subsidize prices in order to gain market share.
Because once they have the market share, they raise prices and/or make the product worse in order to get their money back.
> And users don't really need to be concerned. Just enjoy the low prices. If the service you use goes out of business, it's pretty easy to switch to another.
Well that's the question, isn't it? If the market stays healthy and they don't take the oxygen out of the room and kill the competition and switching costs stay low, then yes by all means enjoy the free ride but plan for its end. But... do plan for it.
They should only be able to do that if they have a considerable moat though?
Right but it means you can't depend on that price remaining low. If it's below cost, the price must rise. That may be important for anyone building businesses on top.
> it's pretty easy to switch to another.
Well that assumes the others have true lower prices.
Well yes but every business owner knows that, or should know that. That's a known business risk. We're all adults here.
If your business fails because you naively assumed that prices you rely on will never go up, then that's on you.
Know that they are buying services below cost? I don't think so.
It's a reasonable bet that compute costs generally trend downwards, and in a competitive market where the key thing you're paying for is compute that should also trend downwards.
That's different to finding out that everyone's been selling things way under the true cost.
To the contrary, of course you should know.
If you know anything about tech businesses, you know that startups being funded by massive amounts of VC keep prices artificially low initially, and that they then rise.
This is tech startup 101 stuff.
It's fine if you're not aware of this as an engineer. But if you're building a business and therefore have a business model where you've identified known risks, there's no excuse for you not to be aware of it.
Is openai selling below cost? Is the cost on azure lower than the true cost? ChatGPT free is obviously run subsidized but are the apis?
Can you tell me the actual cost to Microsoft of a gpt4 API call?
By taking outside money and subsidizing certain market players so that they can sell their goods at below market value (because that is what is happening) you are undermining the market and punishing players who are acting rationally.
This would be absolutely no different than taking tax money and subsidizing certain companies over others to destroy competition and then claiming that it was a market decision.
'We are all adults here' is not a magical phrase that makes cheating OK.
If I launch a business that makes sense given the current pricing of LLMs, and the price then goes up by a factor of ten, my own unit economics may not make sense any more.
There's an Uber comparison here: back when Uber was cheap, some people made life decisions about where they lived and worked that were predicated on being able to afford a commute in an Uber (since public transport doesn't serve a lot of places, at least in the US).
This hurt a lot when the prices went up a few years later!
I'm not sure how well that works if vendors differentiate themselves through proprietary models and backend integrations. As far as I know, Mistral only releases weights for some of its models, with others being proprietary.
https://en.wikipedia.org/wiki/Enshittification
This is how the world gets shittier. It's a whole thing. It's a large part of why things are as bad as they are.
But hey, yeah, don't worry about it, they obviously know what they're doing.
As a consumer it should definitely concern you if you're using poorly financed or unsustainable products. Maybe it's great now but I would rather have a slightly more expensive product that is more stable than one that suddenly had to make the experience a lot worse because they burned a lot of cash and had a down round.
Economically rational for the business owner, maybe, but it inevitably results in eventual enshittification for the consumer. (And Uber isn't a great example of this; its share price has only recently surpassed what it was at IPO 4 years ago, and many companies don't even do that well)
(for example, deepinfra has wizardLM-2-8x22B at $0.65/1M output tokens, compared to $6/1M output tokens for 8x22B by Mistral - and of course Mistral has some more expensive, closed source models that perform better)
For companies that are built off these AI services, there's a very real fear that the prices we're getting on API calls aren't covering the costs of the requests themselves under some assumption that costs will magically go down, or we'll get locked in.
I hope the companies like Mistral and OpenAI know that if they're selling API access below cost they could be the origin of a __lot__ of companies closing and choose to make sure their operating costs are sustainable. (I understand creating new features and new models taxes additional capitol and that's a perfectly fine use of VC money. But selling a $1.00 for $0.90 is dangerous and bad.)