I'm as distant from it all as anyone else, but I can easily believe the narrative that Ilya (et al.) didn't sign up there just to run through a tired page from the tech playbook where they make a better Amazon Alexa with an app store and gift cards and probably Black Friday sales.

As such, it would have been much more of a challenge to shift OpenAI's supposed over-focus on commerce towards a supposed non-profit focus.

He was already the brand, and there likely wouldn't have been a convenient time to remove him from their perspective.

Now we either see a belated - and somewhat erratic - response to all that went before or there is some smoking gun. If there isn't they have just done themselves an immense disservice. Maybe they think they can live without donations now that the commercial ball is rolling downhill fast enough but that only works if you don't damage your brand.

Unless I'm missing something, this stands to reason if you don't work there.

Kinda like how none of us are privy to anything else going on inside the company. We're all speculating in the end, and it's healthy to have an open mind about what's going on without preconceived notions.

It's also possible that the structure of the non-profit/for-profit/operating agreement/etc. just isn't strong enough to achieve the intent and the investors have the strangehold in reality.

If I was invested in the mission statement of OpenAI I don't think I would view the reinstatement of Altman as a good thing, though. Thankfully my interest in all of this is purely entertainment.

Optics matter a lot, even for non-profits, especially for non-profits nominally above a for-profit. Check out their corporate org chart to see how it all hangs together and then it may make more sense:

https://openai.com/our-structure

Each of the boxes where the word 'investor' or 'employee' is present would have standing to sue if the OpenAI board of directors of the non-profit would act against their interests. That may not work out in the long run but in the short term that could be immensely distracting and it could even affect board members privately.

At this level of execution, words are another tool in their toolbox.

But this is where we've come to as a society. I don't think it's a good place.

Unless you're saying my only option is to pick and choose between different sets of people like that?

The thought occurs that it is quite possible that just like humanity is really not ready (we remain concerned) to live with WMD technologies, it is possible that we have again stumbled on another technology that taxes our ethical, moral, educational, political, and economic understanding. We would be far less concerned if we were part of a civilization of generally thoughtful and responsible specimens but we’re not. This is a cynical appraisal of the situation, I realize, but tldr is “it is a systemic problem”.

Making intelligent machines? Colour me disturbed.

Let me ask you this re: "the domain is technical and cognitively demanding" -- do you think Sam Altman (or a Steve Jobs, Peter Thiel, etc.) would pass a software engineer technical interview at e.g. Google? (Not saying those interviews are perfect, they suck, but we'll use that as a gatekeeper for now.). I'm betting the answer is quite strongly "no."

So the selection criterion here is not the ability to perform technically. Unless we're redefining technical. Which leaves us with "intellectually demanding" and "smart", which, well, frankly also applies to lawyers, politicians, etc.

My worry is right now that the farther you go up at any of these organizations, the more the kind of intelligence and skills trends towards the "is good at manipulating and convincing others" kind of spectrum vs the "is good at manipulating and convincing machines" kind of spectrum. And it is into the former that we're concentrating more and more power.

(All that said, it does seem like Sutskever would definitely pass said interview, and he's likely much smarter than I am. But I remain unconvined that that kind of smarts is the kind of smarts that should be making governance-of-humanity decisions)

As terrible as politicians and various "abstract model" applying folks might be, at least they are nominally subject to being voted out of power.

The problem is that our democratic institutions are not doing a good job of producing accountability and legitimacy. Our politics and government bureaucracies are just as corrupted as our private corporations. Sure, in theory we can vote the politicians out of power, but in practice that never happens: Congress has incumbent reelection rates in the 90s.

The unfortunate truth is that nobody is really qualified to be making governance-of-humanity decisions. The real problem is that we have centralized power so much in governments and megacorporations that the few elites at the top end up making decisions that impact everyone even though they aren't qualified to do it. Historically, the only fix for that has been to decentralize power: to give no one the power to make decisions that impact large numbers of people.

Is that true and settled? I only have my anecdotal experience, but in that it is not clear that GPT-3.5 is better than Google's bard for example.

The cost of the computing machinery and the energy costs to run it are actually massive.

Is this a thing? This would be like Switzerland in WWII doing nuclear weapons research to try and get there before the Nazis.

Would that make any difference whatsoever to the Nazis timeframe? No.

I fail to see how the presence of "ethical" AI researchers would slow down in the slightest the bad actors who are certainly out there.

I think U. S. VS Japan is not.necessarily the right model to be thinking here, but U.S. VS U.S.S.R., where we'd like to believe that neither nation would actually launch against the other, but both having the weapon meant they couldn't without risking severe damage in response making it a losing proposition.

That said, I'm sure anyone with an AGI in their pocket/on their side will attempt to use it as a big stick against those that don't, in the Teddy Roosevelt meaning.

Any AI researchers still plodding along at mere human speed are then doomed: they won't be able to catch up even if they manage to reproduce the original breakthrough, since the head start enjoyed by AGI #1 guarantees that its latest iteration is always further along the exponential self-improvement curve and therefore superior to any would-be competitor. Being rational(ists), they give up and welcome their new AI overlord.

And if not, the AI god will surely make them see the error of their ways.

We assume a self improving AI will lead to some runaway intelligence improvement but if it grows at 1% per year or even per month that’s something we can adapt to.

Maybe an ongoing GPU shortage is the only thing that'll save us!

After a couple months, turn on the backdoor.

¯\_(ಠ_ಠ)_/¯

It doesn't make sense with modern AI, where improvement (be it learning or model expansion) is separated from it's normal operation, but I guess some beliefs can persevere very well.

OpenAI positioned itself like that, much the same way Switzerland does in global politics.

Yes, most of the time from the founding until the First World War.

> and openness?

Not sure what sense of "openness" is relevant here.

Establishing a hemispheric sphere of influence was no act of neutrality.

It is in the name OpenAI… not that I think the Swiss are especially transparent, but neither are the USA.

- the Mexican-American War

- Commodore Perry's forced reopening of Japan

- The fact that President Franklin Pierce recognized William Walker's[1] regime as legitimate

- The Spanish-American war

[1]: https://en.wikipedia.org/wiki/William_Walker_(filibuster)

They can use their position to lobby their own government and maybe other governments to introduce laws to govern AI.

There are plenty of people I know from FAANG, now at OpenAI, where they do product design, operation, and DevOps at scale —all complicated, valuable, and worthwhile endeavors in their own right— that don’t need to get in the way of research. They are just the kind of talent that can operate a business with 90% margins to pay for that research.

Could there be requests or internal projects that are less exciting for some people? Sure, but it’s not very hard to set up Chinese Walls, priorities, etc. Every one of those people had to deal with similar concerns at previous companies and would know how to apply the right principles.

If shepherding responsible AGI can be done without a $10B budget in H100, sure… but it seems that scale matters. Having some people in the company sell state-of-the-art solutions to pay for the rest doing cutting-edge, expensive, necessary research isn’t a bad model.

If those separations needed to be re-affirmed, the research formally separated, a board decision approved to share any model from the research arm before it’s commercialized, etc., all that could be implemented within the mission of the entity. Microsoft Research, before them Bell Labs, and many others, have worked like that.

The notion they could just build it and they will come is ludicrous. Sam understood that and was trying to figure out a model that could pay for itself.

Obviously there will be mistakes made along the way. That's how it goes.

Don't forget. ChatGPT has competitors. A lot of them and they're getting pretty good.