News for hackers. Stuff that matters.
Re:OpenAI announces leadership transition by gjsman-1000 on Friday November 17, 2023 @04:14PM (#38310445)
I wonder if the cost of running GPT-3.5 and GPT-4 models at scale turned out to have been astoundingly more expensive than anticipated.

Imagine if you were the CTO of a company, massively underestimated your AWS bill, and presented your board with something enormous. Maybe something like that happened?

Or, if I wanted to speculate to the extremely negative; what if the training and operating costs ballooned to such a degree, that the deal with Microsoft was an attempt to plug the cash hole without having to go to the board requesting an enormous loan? Because the fact that Copilot (edit: previously known as Bing Chat and Bing Image Creator) is free and ChatGPT (edit: and DALL-E 3) are not should be a red flag...

Re:OpenAI announces leadership transition by HankB99 on Friday November 17, 2023 @04:42PM (#38310857)
> Because the fact that Copilot is free ...

I found a tree trial and $10/month $100/year after that. I've asked them to consider a free tier for hobbyists that cannot justify the expense but I'm not holding my breath.

If there is a free tier I did not find, please point me to it!

Re:OpenAI announces leadership transition by dragonwriter on Friday November 17, 2023 @04:37PM (#38310780)
> Imagine if you were the CTO of a company, massively underestimated your AWS bill, and presented your board with something enormous.

Unless there was evidence you had not underestimated but were, e.g., getting a kickback on the cloud costs that you deliverately lowballed in your estimates, they might fire you, but they almost certainly wouldn't put out a press release about it being for your failure to be candid.

That language indicates that the board has a strong belief that there was a major lie to the board or an ongoing pattern of systematic misrepresentation, or a combination.

Re:OpenAI announces leadership transition by synaesthesisx on Friday November 17, 2023 @05:33PM (#38311467)
I don’t think this is necessarily what happened (the CFO would certainly be implicated and it appears they were spared).
Re:OpenAI announces leadership transition by daveguy on Friday November 17, 2023 @04:32PM (#38310702)
"not significantly candid"

and

"no longer has confidence"

points to something more serious than underestimating costs.

Re:OpenAI announces leadership transition by DebtDeflation on Friday November 17, 2023 @04:36PM (#38310774)
Not if the underestimation was to such a degree that it showed they could never even plausibly reach a break even point.
Re:OpenAI announces leadership transition by gjsman-1000 on Friday November 17, 2023 @04:34PM (#38310734)
Other than costs or the allegations by the sister, "not significantly candid" could easily be short for, in my mind:

"not significantly candid in projections for profitability"

"not significantly candid in calculating operation cost increases"

"not significantly candid in how much subscribers are actually using ChatGPT"

etc.

Re:OpenAI announces leadership transition by paulpan on Friday November 17, 2023 @04:46PM (#38310919)
100% this. Firing your well-recognized and seemingly capable CEO means there's a fatal problem, or that he committed something so reprehensible that there was no option but to oust him immediately.

Maybe Sam had been trying to broker a sale of the company without consulting the board first? All speculation until more details are revealed but he must've done something of similar magnitude.

Re:OpenAI announces leadership transition by ARandumGuy on Friday November 17, 2023 @04:58PM (#38311056)
Underestimating costs could be the reasoning if Altman knew the costs would be higher then estimated, and didn't tell the board for an unreasonable amount of time. Burning through a ton of cash for months and not telling the board about it could be enough grounds for this sudden firing.

Of course we have no clue if that's what actually happened. Any conclusions made at this point are complete speculation, and we can't make any conclusions more specific then "this is probably bad news."

Re:OpenAI announces leadership transition by jliptzin on Friday November 17, 2023 @05:08PM (#38311156)
That only makes sense if Altman is the only one with access to the company's financials which obviously can't be the case. No one else noticed truckloads of cash getting flushed down the toilet?
Re:OpenAI announces leadership transition by ARandumGuy on Friday November 17, 2023 @05:23PM (#38311317)
It's certainly possible. Company financials can get very complicated very quickly, and it's possible that Altman was the only person (or one of a small number of people) who had the complete picture.

To be clear, this is only one possible explanation for Altman's firing. And for my money, I don't even think it's the most likely explanation. But right now, those who rely on OpenAI products should prepare for the worst, and this is one of the most existentially threatening possibilities.

Re:OpenAI announces leadership transition by maximus-decimus on Friday November 17, 2023 @11:12PM (#38315055)
I mean... he got fired so if that's what happened, they did notice.
Re:OpenAI announces leadership transition by aenis on Friday November 17, 2023 @04:35PM (#38310754)
The expected value of a leading AI company is probably in hundreds of billions, if not trillions in the foreseeable future. He could be burning billions per month and he'd still be doing great.
Re:OpenAI announces leadership transition by axiak on Friday November 17, 2023 @04:39PM (#38310816)
based on what math? I can see how there can potentially be differentiators here and there to raise value, but I can't see how this statement can be taken prima facie
Re:OpenAI announces leadership transition by sumedh on Friday November 17, 2023 @07:33PM (#38313041)
> based on what math?

Based on future potential. Investors dont know how high will OpenAI go but they know that is going to go high.

Re:OpenAI announces leadership transition by easton on Friday November 17, 2023 @04:29PM (#38310645)
I have to guess the bulk of the cost is being eaten by MS in exchange for the exclusive ability to resell the model.
Re:OpenAI announces leadership transition by swalsh on Friday November 17, 2023 @04:27PM (#38310631)
This is plausible to me, there's no way anyone is making money from my $20 subscription I use ALL DAY LONG.
Re:OpenAI announces leadership transition by coffeebeqn on Friday November 17, 2023 @04:34PM (#38310741)
Is inference really that expensive? Anyway if the price is too low they could easily charge by query
Re:OpenAI announces leadership transition by knicholes on Friday November 17, 2023 @05:13PM (#38311191)
When I was mining with a bunch of RTX 3080s and RTX 3090s, the electricity cost (admittedly) was about $20/month per card. Running a 70B model takes 3-4 cards. Assuming you're pushing these cards to their extreme max, it's going to be $80/mo. Then again, ChatGPT is pretty awesome, and is likely running more than a 70B model (or I think I heard it was running an ensemble of models), so there's at least a ballpark.
Re:OpenAI announces leadership transition by sodality2 on Friday November 17, 2023 @05:27PM (#38311378)
Batched inference makes these calculations hard - roughly takes the same amount of power and time for one inference vs 30 (as i understand it)
Re:OpenAI announces leadership transition by Sebb767 on Friday November 17, 2023 @05:54PM (#38311757)
Datacenters probably do not pay retail rates on electricity, so they might actually run quite a bit cheaper (or more expensive if they use highly available power, but this seems like overkill for pure compute power).
Re:OpenAI announces leadership transition by 015a on Friday November 17, 2023 @07:57PM (#38313340)
Sure, but everything else about a data center is more expensive (real estate, operations people, networking, equipment). There's a reason AWS is so expensive.
Re:OpenAI announces leadership transition by 698969 on Friday November 17, 2023 @11:24PM (#38315147)
Presumably your miner is running 24/7 throughout the month. Not the same for ChatGPT which would answer maybe 10 sessions (with multiple pauses between queries) tops from a single person in a day.
Re:OpenAI announces leadership transition by jstummbillig on Friday November 17, 2023 @04:42PM (#38310864)
Given the arbitrary rate limiting they take the liberty of doing, it's a very deliberate decision and entirely within their control to change at any point.
Re:OpenAI announces leadership transition by sparkling on Friday November 17, 2023 @04:33PM (#38310723)
> Because the fact that Copilot is free and ChatGPT is not should be a red flag...

I'd assume that running a model that only needs to deal with a single programming language (the Copilot plugin knows what kind of code base it is working on) is _a lot_ cheaper than running the "full" ChatGPT 4.

Re:OpenAI announces leadership transition by gjsman-1000 on Friday November 17, 2023 @04:36PM (#38310761)
Sorry for being so precise, but Microsoft renamed Bing Chat to Copilot yesterday, has already rolled it out to all users of Microsoft Edge, and is rolling out a permanent button on the Windows 11 taskbar to access it.

This is what shouldn't add up: Microsoft is literally adding GPT-4, for free, to the Windows 11 taskbar. Can you imagine how much that costs when you look at the GPT-4 API, or ChatGPT's subscription price? Either Microsoft is burning money, or OpenAI agreed to burn money with them. But why would they do that, when that would compromise $20/mo. subscription sales?

Something doesn't financially add up there.

Re:OpenAI announces leadership transition by sparkling on Friday November 17, 2023 @04:37PM (#38310788)
Sorry i assumed you were talking about Github CoPilot (also owned by MS via Github)
Re:OpenAI announces leadership transition by ctc24 on Friday November 17, 2023 @04:45PM (#38310907)
I don't thing there's necessarily anything there. Microsoft might be burning money because they've decided that browser adoption and usage is worth it to them. It doesn't have to involve OpenAI in any way.
Re:OpenAI announces leadership transition by riversflow on Friday November 17, 2023 @04:48PM (#38310936)
You got me excited that Github Copilot was free. Was going to post to tell you it is, in fact, not free. I've been using Bing on Edge browser for a while now, it's super useful! Sad that they rebranded it to Copilot though, "I have been a good Bing :)" will be forever in my memory. [1] RIP Bing, you were a good chat mode.

[1] https://simonwillison.net/2023/Feb/15/bing/

Re:OpenAI announces leadership transition by crucialfelix on Friday November 17, 2023 @05:27PM (#38311369)
Microsoft is pulling browser, search and AI hype mindshare away from Google. That's worth burning money for.
Re:OpenAI announces leadership transition by DSMan195276 on Friday November 17, 2023 @05:14PM (#38311204)
> Imagine if you were the CTO of a company, massively underestimated your AWS bill, and presented your board with something enormous. Maybe something like that happened?

I think the problem there is that the original CTO is now the interim CEO and they are on the board. So while that kind of scenario could make sense, it's a little hard to picture how the CTO would not know about something like that, and if they did you'd presumably not make them CEO afterward.

Re:OpenAI announces leadership transition by JohnFen on Friday November 17, 2023 @08:50PM (#38313893)
> Imagine if you were the CTO of a company, massively underestimated your AWS bill,

Microsoft is investing billions into OpenAI, and much of it is in the form of cloud services. I doubt there was a surprise bill for that sort of thing. But if there was, and Altman is the one who ordered it, I could see the board reacting in a similar way.