Because that's not the actual reason. It looks like a hostile takeover. The "king" of, arguably, the most important company in the world, got kicked out with very little effort. It's pretty extraordinary, and the power shift is extraordinary too.

Kicked out is a bit hyperbole. They don't have their champion anymore but the deal and their minority ownership stake are inked. They still get tech and profits. They might not have a path to owning OpenAI now but that was a problem a few years down the road. They can also invest in Altmans new thing and poach OpenAI talent to bolster their internal AI research which is probably going to get a massive funding boost.

The PR hit will be bad for a few days. Good time to buy MS stock on discount but this won't matter in a year or two.

The board firing a CEO is hardly a hostile takeover.

Boards have exactly one job.

(It's firing the CEO, if anyone wasn't aware.)

If you fire one board member (Altman) and remove another from the board (Brockman) it's not exactly friendly either

Firing generally isn't friendly, but no one "took over." The people who had the power exercised it. Maybe they shouldn't have, I feel no compulsion to argue on their behalf, but calling it a "takeover" isn't correct.

I think when people say "takeover" or "coup" it's because they want to convey their view of the moral character of events, that they believe it was an improper decision. But it muddies the waters and I wish they'd be more direct. "It's a coup" is a criticism of how things happened, but the substantive disagreements are actually about that it happened and why it happened.

I see lots of polarized debate any time something AI safety related comes up, so I just don't really believe that most people would feel differently if the same thing happened but the corporate structure was more conventional, or if Brockman's board seat happened to be occupied by someone who was sympathetic to ousting Altman.

"It's a coup" is loaded language and lets the user insinuate their position without actually explaining and justifying it.

Maybe someone from higher up called the board?

ChatGPT-5?

No he didn't fire Sam over AI safety concerns. That's completely made up by people in the twittersphere. The only thing we know is that the board said the reason was that he lied to the board. The guardian[1] reported that he was working on a new startup[1] and that staff had been told it was due to a breakdown in communication and not to do with anything regarding safety, security, malfeasance or a bunch of other things.

[1] https://www.theguardian.com/technology/2023/nov/18/earthquak...

The Atlantic Article makes it pretty clear that the fast growth of the commercial business was giving Ilya too few resources and too little time to do the safety work he wanted to do: https://archive.ph/UjqmQ

I can buy that they fired him over a disagreement about strategy (ie are we going too fast/we are concentrating on the wrong things etc), because in general of course board members get fired if they can't work together on a common strategy. But the narrative that has taken over lots of places at the weekend is more along the lines of he got fired because they had created a sentient AI and Ilya was worried about it. That just makes no sense to me.

Additionally, no-one (not insiders at OpenAI and certainly not a journalist) other than people in those conversations actually knows what happened, and noone other than Ilya actually knows why he did what he did. Everyone else is relying on rumor and heresay. For sure the closer people are to the matter the more insight they are likely to have, but noone who wasn't in the room actually knows.

Spoken to a bunch of folk at OpenAI, it really does seem to be regarding safety. Ilya was extremely worried, did not like the idea of GPT’s as users can train AI’s to do arbitrarily harmful stuff.

> Did he really fire Sam over "AI safety" concerns? How is that remotely rational.

Not rational iff (and unlike Sustkever, Hinton, Bengio) you are not a "doomer" / "decel". Ilya's very vocal and on record that he suspects there may be "something else" going on with these models. He and DeepMind claim AlphaGo is already AGI (correction: ASI) in a very narrow domain (https://www.arxiv-vanity.com/papers/2311.02462/). Ilya particularly predicts it is a given that Neural Networks would achieve broad AGI (superintelligence) before alignment is figured out, unless researchers start putting more resources in it.

(like LeCun, I am not a doomer; but I am also not Hinton to know any better)

> AGI in a very narrow domain

The definition of AGI always puzzles me, because "G" in AGI is general, and the word certainly don't play well w/ "narrow". AGI is a new buzzword I guess.

Well there's nothing narrow about sota LLMs. The main hinge is just competence.

i think the guy you're replying to misunderstood the article he's alluding to though. They don't claim anything about a narrow agi

Can you please share the sources for Ilyas views?

For what it's worth, the MIT Technology Review these days is considered to be closer to a "tech tabloid" than an actual news source. I personally would find it hard to believe (on gut instinct, nothing empirical) that AGI has been achieved before we have a general playbook for domain-specific SotA models. And I'm of the 'faction' that AGI can't come soon enough.

> hard to believe (on gut instinct, nothing empirical) that AGI has been achieved before we have a general playbook for domain-specific SotA models

Ilya is pretty serious about alignment (precisely?) due to his gut instinct: https://www.youtube.com/watch?v=Ft0gTO2K85A (2 Nov 2023)

I don't doubt he's serious in what he believes. I respect Ilya greatly as a researcher. Do you have notes or a time-point for me to listen to in that podcast? I bemoan the trend toward podcasts-as-references - even a time-point reference (or even multiple!) to the transcript would be greatly desirable!

> "[Artificial General Intelligence] in a very narrow domain."

Which is it?

I think the guy read the paper he linked the wrong way. The paper explicitly separates "narrow" and "AGI" types where AlphaGo is in the virtuoso bracket for narrow AI, and ChatGPT is in the "emerging" bracket for "general" AI. Only thing it puts to be AGI is few levels up from virtuoso, but in the "general" type.

Read the paper linked above, and if you don't agree that's okay. There are many who don't.

Has anyone written a response to this paper? Their main gist is to try to define AGI empirically using only what is measurable.

Check it again, I think you might have misread the thing. It categorizes things in a way that clearly separates AlphaGO from even shooting towards "AGI". The "General" part of AGI can't really be skipped or words don't make any sense anymore.

Ah, gotcha; I meant "superintelligence" (which is ASI and not AGI).

To shine some light on the true nature of the "AI safety tribe" aspects I highly recommend reading the other top HN post / article : https://archive.is/Vqjpr

It might be because of AI safety, but I think it's more likely because Sam was executing plans without informing the board, such as making deals with outside companies, allocating funds to profit-oriented products and making announcements about them, and so on. Perhaps he also wanted to reduce investment in the alignment research that Ilya considered important. Hopefully we'll learn the truth soon, though I suspect that it involves confidential deals with other companies and that's why we haven't heard anything.

It's to do with a tribe in openAI that believes ai will take over the world in the next 10 years so we need to spend much of our efforts towards that goal. What that translates to is strong prompt censorship and automated tools to ban those who keep asking things we don't want you to ask.

Sam has been agreeing with this group and using this as the reason to go commercial to provide funding for that goal. The problem is these new products are coming too fast and taking resources which affects the resources they can use for safety training.

This group never wanted to release chatGPT but were forced to because a rival company made up of ex openAI employees were going to release their own version. To the safety group things have been getting worse since that release.

Sam is smart enough to use the safety group's fear against them. They finally clued in.

OpenAI never wanted to give us chatGPT. Their hands were forced by a rival and Sam and the board made a decision that brought in the next breakthrough. From that point things snowballed. Sam knew he needed to run before bigger players moved in. It became too obvious after devday that the safety team would never be able to catch up and they pulled the breaks.

OpenAI's vision of a safe AI has turned into a vision of human censorship rather than protecting society from a rogue AI with the power to harm.

> What that translates to is strong prompt censorship and automated tools to ban those who keep asking things we don't want you to ask.

...which several subreddits dedicated to LLM porn or trolling could tell you is both mostly pointless and also blocks a ton of stuff you could find on any high school nerd's bookshelf as "unsafe".

If it really was about “safety” then why wouldn’t Ilya have made some statement about opening the details of their model at least to some independent researchers under some tight controls. This is what makes it look like a simple power grab, the board has said absolutely nothing about what actions they would take to move toward a safer model of development.

Because they want to slow down further research which would push AGI closer until the safety/alignment aspect can catch up.

But if you really cared about that why would you be so opaque on everything. Usually people with strong conviction try to convince other people of that conviction. For a non profit that is supposedly acting in the interests of all mankind, they aren't actually telling us shit. Transparency is pretty much the first thing everybody does who actually cares about ethics and social responsibilities.

Ilya might be a believer in what Eliezer Yudkowsky is saying, which is that opacity is safer.

https://x.com/esyudkowsky/status/1725630614723084627?s=46

Mr. Yudkowsky is a controversial figure who a lot of AI Safety people tend to distance themselves from because he has a tendency to exaggerate for rhetorical effect, which means that he ends up “preaching to the choir” while pushing away people in the general public who might be open to learning about AI x-risk scenarios but haven’t made up their mind yet.

But I’d like to publicly thank Mr. Yudkowsky and say that even if he has fallen out of favor for being too extreme (kind of like Richard Stallman), Mr. Yudkowsky was one of the 3 or 4 people most central to creating the field of AI safety, and without him, OpenAI and Anthropic would not exist.

wouldn’t that mean that they’ll just be left behind and it won’t matter what their goal is?

It depends how far ahead they currently are.

That's because it's not about safety, it's about ego, vanity, and delusion.