Will people love ChatGPT et al just as much if OpenAI have to charge what it costs them to buy and run all the GPUs? Maybe, but it's absolutely not certain.
If they "went defunct" tomorrow then the people who just invested US$6bn and lost every penny probably would not agree with your assessment that it "ended well".
It’s less (gross) expensive for inference, since it takes less time, but the cost of that time (per second) is the same as training.
We can do the math. GPT-4o can emit about 70 tokens a second. API pricing is $10/million for output tokens and $2.5/million for input tokens.
Assuming a workload where inputs tokens are 10:1 with output tokens, and that I can generate continuous load (constantly generating tokens). I'll end up paying $210/day in API fees, or $76,650 in a year.
Let's assume the hardware required to service this load is a rack of 8 H100s (probably not accurate, but likely in the ballpark.). That cost $240k.
So the hardware would pay for itself in 3 years. It probably has a service life of about double that.
Of course we have to consider energy too. Each H100 is 700watts, meaning our rack is 5.6 kilowatts, so we're looking at about 49 megawatt-hours to operate for the year. Let's assume they pay wholesale electricity prices of $50/mwh (not unreasonable), and you're looking at a ~$2,500 annual energy bill.
So there's no reason to think that inference alone isn't a profitable business.
The issue is OpenAI is not just selling inference.
Though I wouldn’t be surprised if there were some hidden costs that are hard for us to account for due to the sheer amount of traffic they must be getting on an hourly basis.
It's not unusual for a startup to not be profitable, and they're obviously not as the company doesn't make a profit, but I'm not sure why isolating one aspect of their business and declaring it profitable would justify the idea that this company is inevitably a good investment "even if the company went defunct tomorrow".
Perhaps you meant "win" in the sense of "being influential" or something, but I'm pretty sure the people who invested billions of dollars use definitions that involve more concrete returns on their investment.
I was responding to someone upthread suggesting that they were running even inference at a loss.
I'm willing to bet that if you swapped out GPT with Claude, Gemini or Llama under the hood 95% of their users wouldn't even notice. LLMs are fast becoming a commodity. The differentiating factor is simply how many latest NVIDIA GPUs the company owns.
And even otherwise, people loving a product isn't what makes a company successful. People loved WeWork as well. Ultimately what matters is the quarterly financial statement. OpenAI is burning an incredible amount of money on training newer models and serving every query, and that's not changing anytime soon.
You can say exactly the same about Google and Bing (or any other search engines), yet Google search is still dominant. Execution, market perception, brand recognition, momentum are also important factors, not to mention talent and funding.
Not everyone who wants to invest, can invest in this round. You may bet the investors are wrong, but they put money where their mouth is. Microsoft participate, even though they already invested $13b.
How is A16Z's massive crypto fund doing again? And what about Softbank's other big bets?
The aspect of corporate usage where OpenAI seems to be ahead is in direct enterprise subscriptions to ChatGPT.
It seems that me most likely outcome is that they have one replaceable product against many and few options to get return commensurate with valuation.
My guess is that investors are are making a calculated bet. 90% chance the company become irrelevant, 10% chance it has a major breakthrough and somehow throws up a moat to prevent everyone else from doing the same.
That said, I have no clue what confidential information they are showing to investors. For all we know, they are being shown super human intelligence behind closed doors.
This seems to be the "crypto is about to replace fiat for buying day to day goods/services" statement of this hype cycle. I've been hearing it at least since gpt-2 that the secret next iteration will change everything. That was actually probably most true with 2 given how much of a step function improvement 3 + chatGPT were.
If that were the case, I wonder why Apple passed on this investment.
also, for my use case I eventually found it’s faster and easier and less frustrating to write code myself (empowering) and not get sucked into repeatedly reminding AI of my intent and asking it to fix bugs and divergences (disempowering)
Plus, you can find alternatives now for the random edge cases where you do actually want to chat with an out of date version of the docs, which don’t train on user input.
I recommend we all “hard pass” on OpenAI, Anthropic, Google, basically anyone who’s got prohibitions on competition while simultaneously training their competing intelligence on your input. Eventually these things are going to wreck our knowledge work economy and it seems like a form of economic self-harm to knowingly contribute to outsource our knowledge work to externals…
The git repo and review history for any large project is probably more helpful for training a model than anything, including people using the model to write code.
It sounds like you are happy to not use LLMs. I’m the opposite way; code is a means to an end. If an LLM can help smooth the road to the end result, I’m happy to take the smooth road.
Refusing to learn the new tool won’t keep it from getting made. I really don’t think that code writers are going to influence it that much. The training data is already out there.
...yet they struggle to find productive applications, shamefully hide their training data and can't substantiate their claims of superhuman capability. You could have said the same thing about Bitcoin and been technically correct, but society as a whole moved in a different direction. It's really not that big of a stretch to imagine a world where LLM capability plateaus and OpenAI's value goes down the toilet.
There is simply no evidence for the sort of scaling Sam Altman insists is possible. No preliminary research has confirmed it is around the corner, and in fact tends to suggest the opposite of what OpenAI claims is possible. It's not nuclear fusion or commercial supersonic flight - it's a pipe-dream from start to finish.
They are not the only ones with the product. They don't have a moat. They are marginally better than competing models, at best. There is no moat into LLMs unless you happen to find the secret formula for super intelligence, hope nobody else finds it, and lock all of your R&D in the Moria mines so they don't go working and building it elsewhere.
There is no moat here. I can't believe so many intelligent people, even on HN cannot grasp it.
I mean, what exactly do you see happening? The have a product people love and practically incalculable upside potential. They may or may not end up the winners, but I see no scenario in which it "doesn't end well". It's already "well", even if the company went defunct tomorrow.
>that clause with that timeline seems a bit risky.
I'm 99% certain that OpenAI drove the terms of this investment round, they weren't out there hat in hand begging. Debt is just another way to finance a company, cant really say it's better or worse.