https://www.wheresyoured.at/oai_docs/
However, it needs to be said that he received those numbers. I personally have quite a few issues with him, but there's no reason to doubt his journalistic integrity. Because of that, I believe he reports truthfully on data he receives by informants.
Additionally, none of the frontier models actually publicly talks about inference costs in anything but broad, "let's just forget that"-like takes. Which does not exactly spark confidence.
I'm eagerly awaiting anthropic's public disclosure of their financial details. That should be rather interesting in any case and finally put the inference-discussion to rest.
I'll gladly tell my customers all the most glorious stuff about AI and big tech while spending a significant chunk of the money they pay me on supporting AI-/tech-counterculture, such as doctorow, zitron and quite a few other writers, journalists and activists.
It's the old "you live in a society" counter-point against anti-capitalist activism. Needing to make ends meet does not imply that your points or principles are meaningless, it just implies that you have no interest in being homeless and that way losing your chance to actually change things.
So that's fine to me. But: I stated it for a reason, because I know others don't agree. I, personally, consider him trustworthy. You do not, and that's fine. I suspect we both await anthropic's Z.1, which will be able to settle a big chunk of the debate.
If he is right, the numbers will show it.
He was also right about AI-video and sora in particular being a fundamentally flawed idea.
He was also right about the dangers and problems with the general inaccuracy of LLMs and people relying on it.
Also about the expected triggering of ROI-checking in companies, such as Uber is doing now. His prediction is, ROI is negative. And I'm awaiting the society's consensus on that.
The general direction seems correct to me. He's not a technical guy and does not have the knowledge to critique models on a factual basis. I do wish he'd just focus on the stuff he _does_ know about, which is the financial side of things.
He is a much needed counterweight to the unhealthy hype going around, imho.
He specifically predicted that AI videos have plateaued in 2024 which is egregiously wrong.
> He was also right about the dangers and problems with the general inaccuracy of LLMs and people relying on it.
He specifically predicted that accuracy won't increase but accuracy has increased over the time significantly to the point where you can't get it to say anything inaccurate using the reasoning models.
> Also about the expected triggering of ROI-checking in companies, such as Uber is doing now. His prediction is, ROI is negative. And I'm awaiting the society's consensus on that.
The whole Uber skepticism is a good point because all of those people were wrong and Uber is profitable now.
You didn't address my other criticisms - he claimed that revenue would drop in 2024 and it skyrocketed. He claimed that users weren't interested in ChatGPT but now it has a billion users (6x jump).
1. What data?
2. Intuitions = vibes.
Vibes are bad when used against you, but good when used in your favor.
Come on :-)))
But if you don't believe me, lets have a bet based on what the IPO filings show?
A better way to model this, since you seem interested is the following:
How much would it cost you to start such a service for, say, 10k users?
Any other internet service has had virtually Zero cost, $0. Google, Facebook, youtube, Wikipedia, you name it. They all went into the dumpster to pick up a thrown away desktop computer, and they could serve up towards 100k if not a million users.
How much would it cost you to serve, say, 10k simultaneous users with a SOTA model? And if you wanted to go cash positive after a year, how much would each user have to pay?
My post has this same argument - we have multiple third party companies running open weight models. They are obviously not subsidised. And people are willing to pay for it. And these models are as good as the SOTA models from last year. So this kinda proves my point that SOTA is sustainable.
What hardware is needed, how much of it, cooling, and what does it all cost you?
Or are you saying I can take my old desktop and serve Deepseek v3.2 to 10k users simultaneously and it would cost me about $1 per megatoken?
Obviously they are not subsidised, do you disagree? If you agree, they have a way to price it at a point that people wanna pay for it and also they aren't losing money.
So there's nothing inherent about inference that makes it too costly or whatever.
It said: upfront investment: $3M to $6M.
Customers should pay $25k per month.
Checks out
>The "$25k per month" figure is almost certainly the result of ChatGPT making assumptions, not a fact derived from any known business model.
https://chatgpt.com/share/6a28193b-6ec0-8333-a1af-d07e8d89ef...
Your whole calculation is also ridiculous - I think you assumed what revenue per month is required to pay off hardware within a year? Why would I use hardware within a year?
I would suggest consulting with ChatGPT and coming up with a better and more coherent argument.
Minimum upfront: about $15M
Comfortable upfront: $20M–$25M
Monthly revenue needed: $900k–$1.5M
Required price per 10k individual customers: $99–$149/month
API-equivalent output price: usually $8–$20/M output tokens, unless utilization is very high.
I try to do some napkin math of what it takes to start a company serving an LLM to customers. I thought maybe 10,000 users sounded like a reasonable number.
I'd like also to compare it to traditional Internet companies, e.g. Twitter. I'd guess Twitter with 10k users would cost me literally a dumpster dive, so essentially no cost.
If I were to start an LLM company, what would my initial investment look like?
Turns out around 15+ million dollars assuming I would get 10,000 users willing to pay 200$/month.
I just don't see how the numbers you show add up.
> Obviously they are not subsidised, do you disagree? If you agree, they have a way to price it at a point that people wanna pay for it and also they aren't losing money.
> So there's nothing inherent about inference that makes it too costly or whatever.
Do we have audited GAAP financial data for any of these companies? If we don't, all these are... vibes, man.
But isn't it okay to suggest that random 3rd party hosting companies are ... not losing money? Why?
I bet that within 5 years they will be sold for scrap to bigger companies and will become divisions inside them.
Statement: margins API prices of all models are greater than 10% in Anthropic.
2. Rephrasing your statement:
At June 2026 prices, profit margins for all Anthropic models are >10%.
Even so, fine, I'll take that bet. Anthropic inference prices are still marginally subsdizided. Once they're public they will hike their API prices several times over the next 24 months. Even that might not save them, because when we take all their expenses into account, they will probably need to raise prices 2-3x compared to their June 2026 prices.
Ok lets state the bet like this: 5 months after their IPO, it will be clear that their API prices still have greater than 10% margin.
This is often repeated but comes from ignorance mostly. You have * zero * reason to believe inference is costly other than just vibes. If you go by data and intuitions - the margins are high.
This kind of thinking really reinforces my belief that people have no idea and are using this whole [AI is not profitable and too costly] thing as a cathartic way to deal with immense progress.