But when you're profit-seeking company is owned by a non-profit with a public mission, that trajectory might end up pointed the wrong way. The Dev Day announcements, and especially the marketplace, can be seen as suggesting that's exactly what was happening at OpenAI.

I don't think everyone there wants them to be selling cool LLM toys, especially not on a "make fast and break things" approach and with an ecosystem of startup hackers operationalizing it. (Wisely or not) I think they want to be shepherding responsible AGI before someone else does so irresponsibly.

I'm as distant from it all as anyone else, but I can easily believe the narrative that Ilya (et al.) didn't sign up there just to run through a tired page from the tech playbook where they make a better Amazon Alexa with an app store and gift cards and probably Black Friday sales.

As such, it would have been much more of a challenge to shift OpenAI's supposed over-focus on commerce towards a supposed non-profit focus.

He was already the brand, and there likely wouldn't have been a convenient time to remove him from their perspective.

Now we either see a belated - and somewhat erratic - response to all that went before or there is some smoking gun. If there isn't they have just done themselves an immense disservice. Maybe they think they can live without donations now that the commercial ball is rolling downhill fast enough but that only works if you don't damage your brand.

Unless I'm missing something, this stands to reason if you don't work there.

Kinda like how none of us are privy to anything else going on inside the company. We're all speculating in the end, and it's healthy to have an open mind about what's going on without preconceived notions.

It's also possible that the structure of the non-profit/for-profit/operating agreement/etc. just isn't strong enough to achieve the intent and the investors have the strangehold in reality.

If I was invested in the mission statement of OpenAI I don't think I would view the reinstatement of Altman as a good thing, though. Thankfully my interest in all of this is purely entertainment.

Optics matter a lot, even for non-profits, especially for non-profits nominally above a for-profit. Check out their corporate org chart to see how it all hangs together and then it may make more sense:

https://openai.com/our-structure

Each of the boxes where the word 'investor' or 'employee' is present would have standing to sue if the OpenAI board of directors of the non-profit would act against their interests. That may not work out in the long run but in the short term that could be immensely distracting and it could even affect board members privately.

At this level of execution, words are another tool in their toolbox.

But this is where we've come to as a society. I don't think it's a good place.

Unless you're saying my only option is to pick and choose between different sets of people like that?

The thought occurs that it is quite possible that just like humanity is really not ready (we remain concerned) to live with WMD technologies, it is possible that we have again stumbled on another technology that taxes our ethical, moral, educational, political, and economic understanding. We would be far less concerned if we were part of a civilization of generally thoughtful and responsible specimens but we’re not. This is a cynical appraisal of the situation, I realize, but tldr is “it is a systemic problem”.

Making intelligent machines? Colour me disturbed.

Let me ask you this re: "the domain is technical and cognitively demanding" -- do you think Sam Altman (or a Steve Jobs, Peter Thiel, etc.) would pass a software engineer technical interview at e.g. Google? (Not saying those interviews are perfect, they suck, but we'll use that as a gatekeeper for now.). I'm betting the answer is quite strongly "no."

So the selection criterion here is not the ability to perform technically. Unless we're redefining technical. Which leaves us with "intellectually demanding" and "smart", which, well, frankly also applies to lawyers, politicians, etc.

My worry is right now that the farther you go up at any of these organizations, the more the kind of intelligence and skills trends towards the "is good at manipulating and convincing others" kind of spectrum vs the "is good at manipulating and convincing machines" kind of spectrum. And it is into the former that we're concentrating more and more power.

(All that said, it does seem like Sutskever would definitely pass said interview, and he's likely much smarter than I am. But I remain unconvined that that kind of smarts is the kind of smarts that should be making governance-of-humanity decisions)

As terrible as politicians and various "abstract model" applying folks might be, at least they are nominally subject to being voted out of power.

The problem is that our democratic institutions are not doing a good job of producing accountability and legitimacy. Our politics and government bureaucracies are just as corrupted as our private corporations. Sure, in theory we can vote the politicians out of power, but in practice that never happens: Congress has incumbent reelection rates in the 90s.

The unfortunate truth is that nobody is really qualified to be making governance-of-humanity decisions. The real problem is that we have centralized power so much in governments and megacorporations that the few elites at the top end up making decisions that impact everyone even though they aren't qualified to do it. Historically, the only fix for that has been to decentralize power: to give no one the power to make decisions that impact large numbers of people.

Is that true and settled? I only have my anecdotal experience, but in that it is not clear that GPT-3.5 is better than Google's bard for example.

The cost of the computing machinery and the energy costs to run it are actually massive.

Is this a thing? This would be like Switzerland in WWII doing nuclear weapons research to try and get there before the Nazis.

Would that make any difference whatsoever to the Nazis timeframe? No.

I fail to see how the presence of "ethical" AI researchers would slow down in the slightest the bad actors who are certainly out there.

I think U. S. VS Japan is not.necessarily the right model to be thinking here, but U.S. VS U.S.S.R., where we'd like to believe that neither nation would actually launch against the other, but both having the weapon meant they couldn't without risking severe damage in response making it a losing proposition.

That said, I'm sure anyone with an AGI in their pocket/on their side will attempt to use it as a big stick against those that don't, in the Teddy Roosevelt meaning.

Any AI researchers still plodding along at mere human speed are then doomed: they won't be able to catch up even if they manage to reproduce the original breakthrough, since the head start enjoyed by AGI #1 guarantees that its latest iteration is always further along the exponential self-improvement curve and therefore superior to any would-be competitor. Being rational(ists), they give up and welcome their new AI overlord.

And if not, the AI god will surely make them see the error of their ways.

We assume a self improving AI will lead to some runaway intelligence improvement but if it grows at 1% per year or even per month that’s something we can adapt to.

Maybe an ongoing GPU shortage is the only thing that'll save us!

After a couple months, turn on the backdoor.

¯\_(ಠ_ಠ)_/¯

It doesn't make sense with modern AI, where improvement (be it learning or model expansion) is separated from it's normal operation, but I guess some beliefs can persevere very well.

OpenAI positioned itself like that, much the same way Switzerland does in global politics.

Yes, most of the time from the founding until the First World War.

> and openness?

Not sure what sense of "openness" is relevant here.

Establishing a hemispheric sphere of influence was no act of neutrality.

It is in the name OpenAI… not that I think the Swiss are especially transparent, but neither are the USA.

- the Mexican-American War

- Commodore Perry's forced reopening of Japan

- The fact that President Franklin Pierce recognized William Walker's[1] regime as legitimate

- The Spanish-American war

[1]: https://en.wikipedia.org/wiki/William_Walker_(filibuster)

They can use their position to lobby their own government and maybe other governments to introduce laws to govern AI.

There are plenty of people I know from FAANG, now at OpenAI, where they do product design, operation, and DevOps at scale —all complicated, valuable, and worthwhile endeavors in their own right— that don’t need to get in the way of research. They are just the kind of talent that can operate a business with 90% margins to pay for that research.

Could there be requests or internal projects that are less exciting for some people? Sure, but it’s not very hard to set up Chinese Walls, priorities, etc. Every one of those people had to deal with similar concerns at previous companies and would know how to apply the right principles.

If shepherding responsible AGI can be done without a $10B budget in H100, sure… but it seems that scale matters. Having some people in the company sell state-of-the-art solutions to pay for the rest doing cutting-edge, expensive, necessary research isn’t a bad model.

If those separations needed to be re-affirmed, the research formally separated, a board decision approved to share any model from the research arm before it’s commercialized, etc., all that could be implemented within the mission of the entity. Microsoft Research, before them Bell Labs, and many others, have worked like that.

The notion they could just build it and they will come is ludicrous. Sam understood that and was trying to figure out a model that could pay for itself.

Obviously there will be mistakes made along the way. That's how it goes.

Don't forget. ChatGPT has competitors. A lot of them and they're getting pretty good.

It's such a small cohort that when someone doesn't completely blow it, they're immediately deemed as geniuses.

Give someone billions of dollars and hundreds of brilliant engineers, researchers and many will make it work. But only a few ever get the chance, so this happens.

They don't do any of the work. They just take the credit.

Now imagine the weekend for those fired and those who quit OpenAI: you know they are talking together as a group, and meeting with others offering them billions to make a pure commercial new AI company.

An Oscar worthy film could be made about them in this weekend.

Sad thing so, if they find enough people to continue investing, they will ultimately launch a product, most likely the early employees and founders will sell of their shares, become instant millionaires in the three figures and be hailed as thebtrue geniuses in their field... What an utter shit show that was...

Funnily enough I also worked for a CEO that hit the lottery with timing and became a millionaire. He then drank his own kool-aid and thought he was some sort of Steve Jobs. Of course he never managed to build anything afterwards. But he kept making a shit ton of money, without a doubt.

After they get one position in that echelon, they can keep failing upwards ad nauseam.

I don't get the cult part though. It's so easy to see they're not even close to the geniuses they pretend to be. Just look at the recent SBF debacle. It's pathetic how folks fall for this.

Getting a company to that size is a lot.

Well, I'm happy to work with adults again, in a sane environment with people that known their job. It was a very, very useful experience so, and I wouldn't miss it.

And many times even when they do blow it, it's handwaved away as being something outside of their control, so let's give them another shot.

The idea that success at it shouldn’t be grounds for the genius label is absurd.

And also destroy more. The line between is very thin and littered with landmines.

Of course it's not a boolean, it's a spectrum. But the point remains: valuing lucky rich people with connections as geniuses because they are lucky, rich and connected is nonsensical to me

So by your standard SBF is an absolute genius.

Some founders don’t do much, and some are violently toxic (Lord knows I worked for many), but it’s rarely how they gather big financing rounds. At least, the terrible ones I know rarely did.

CEOs… I’ve seen people coast from Consulting or Audit into very mediocre careers, so I wouldn’t understand if Conway defended them as a class. The Cult for Founders has problems (for the reasons you point out, especially those who keep looking for ‘technical cofounders’ for years), but it’s not as blatantly unfounded.

The worry now is that the approach is going to be more of controlling access to just researchers who are trusted to be “safe”.

A non-profit immediately makes the values of OpenAI's PPUs (their spin on RSUs) to zero. Employees will be losing out of life changing sums on money.

I think it's likely that we're going to find out Sam and others are just talented tech evangelists/hucksters and that justifiably worries a lot of people currently operating in the tech community.

Unless someone is truly well versed in the production of something, they latch on to the most public facing aspect of that production and the person at the highest level of authority (to them, even though directors and CEOs often have to answer to others as well)

That’s not to say they don’t have an outsized individual effect, but it’s rare their greatness is solo

I have often thought that we don't have enough information on how film directors operate, as I feel it could yield a lot of insight. There's probably a reason why many film directors don't hit their stride until late 30s and 40s, presumably because it takes those one or two decades to build the appropriate experience and knowledge.

https://en.wikipedia.org/wiki/Zachary_Quinto

Is pretty close to that image.

Isn’t that essentially the job of a film producer? You do see a lot of productions where there’s a ton of executive producer titles given out as almost a vanity position.

It kind of feels to me like you’re describing the way the industry works from an outsiders view since it doesn’t match the actual workings of any of the productions I’ve worked on.

the shoots are only a portion of production. You have significant pre production and post production time.

A producer is closer in role to a CFO or investor , depending on the production since it’s a relatively vague term.

Your original comment mentioned auteurs, which is what influenced the type of film director I was thinking of, which often are also producers and even writers and editors on their own films. To my knowledge, I am not aware of any famous CEOs that fit the style and breadth of these directors, as the CEO is almost never actually doing anything nor even knowledgeable in the things they're tasking others to do.

So to summarize, there are auteur directors but not CEOs, despite many thinking there are auteur CEOs. If there are, they are certainly none of the famous ones and are likely running smaller companies. I generally think of CEOs as completely replaceable, and usually the only reason one stands out is that they don't run the business into the ground or have a cult of personality surrounding them. If you take away an auteur director from their project, it will never materialize into anything remotely close to what was to be.

Thinking of directors with distinctive styles like Hitchcock, Fincher, Spielberg, Wes Anderson etc… they’re maybe some who have a much larger influence than others, but I think there are very few projects that depend on that specific director being involved to make a good film, just not the exact film that was made. The best of them know exactly how to lean on their crew and maximize their results.

Taking that kind of influence, I’d say there have certainly been CEOs of that calibre. Steve Jobs instantly springs to mind. Apple and Pixar definitely continued and had great success even after he left them/this world, but he had such an outsize influence that it’s hard not to call him an auteur by the same standards.

In fact most movie productions don’t even have the director involved with story. Many are just directors for hire and assigned by the studio to scripts.

Of course there are exceptions but they are the rarities.

And the big reason directors don’t hit their big strides till later is movies take a long time to make and it’s hard to work your way up there unless you start as an indie darling. But even as an indie, let’s say you start at 20, your film would likely come out by the time you’re 22-24 based on average production times. You’d only be able to do 2 or 3 films by 30, and in many cases would be put on studio assignments till you get enough clout to do what you want. And with that clout comes the ability to hire better people to manage the other aspects of your shoot.

Again, I think this is people prescribing to auteur theory. It takes a huge number of people to pull off a film, and the film director is rarely well versed in most. Much like a CEO, they delegate and give their opinion but few extend beyond that.

For reference I’ve worked on multiple blockbuster films, many superhero projects, some very prestigious directors and many not. The biggest indicator that a director is versed in other domains is if they worked in it to some degree before being a director. That’s where directors like Fincher excel and many others don’t

In this case, tons of people already have resigned from OpenAI. Sam Altman seems very likely to start a rival company. This is a huge decision and will have massive consequences for the company and their product area.

Sam Altman has been an objectively successful leader of OpenAI.

Everyone has their flaws, and I'm more of a Sam Altman hater than a fan, but even I have to admit he led OpenAI to great success. He didn't do most of the actual work but he did create the company and he did lead it to where it is today.

Personally, If I had stock in OpenAI I'd be selling it right now. The odds of someone else doing as good a job is low. And the odds of him out-competing OpenAI is high.

I'm not sure this is actually the case, even ignoring the non-profit charter and the for-profit being beholden to it.

We know that OpenAI has been the talk of the town, we know that there is quite a bit of revenue, and that Microsoft invested heavily. What we don't know is if the strategy being pursued ever had any chance of being profitable.

Decades-long runways with hope that there is a point where profitability will come and at a level where all the investment was worth it is a pretty common operating strategy for the type of company Altman has worked with and invested in, but it is less clear to me that this is viable for this sort of setup, or perhaps at all - money isn't nearly as cheap as it was a decade ago.

What makes a for-profit startup successful isn't necessarily what makes a for-profit LLC with an operating agreement that makes it beholden to the charter of a non-profit parent organization successful.

In what way, exactly? ChatGPT would have been built regardless of whether he was there or not. It's not like he knows how to put a transformer pipeline together. The success of OpenAI's product rests on its scientists and engineers, not the CEO, and certainly not a non-technical one like Mr. Altman.

There are much larger armies of highly qualified scientists and engineers at Google, Microsoft, and other companies and none of them created ChatGPT. They still haven't been able to even fully copy it with years of effort, unlimited budgets, and the advantage of knowing exactly what they're trying to build.

It should really give you pause to consider why it happened at OpenAI and not elsewhere. The answer is that the CEO of OpenAI created the incentives, hiring, funding, vision, support, and direction that made ChatGPT happen.

Leadership makes all the difference in the world.

This visionary bullshit is exactly that, bullshit.

I'm absolutely not comparing Sam Altman to any of these leaders, but just to illustrate how much vision and leadership does matter. Consider how stupid these statements sound:

"Jesus didn't build any churches, those were all built by brick layers and carpenters!"

"Pharaohs didn't build a single pyramid, those were all built by artists and workers!"

"Abraham Lincoln didn't free any slaves, he didn't break the chains of a single slave, that was all done by blacksmiths!"

"Martin Luther King Jr. didn't radically improve civil rights, he never passed a single law, that was all done by lawmakers!"

"Sam Altman didn't build ChatGPT, he didn't create a single ML pipeline, it was all done by engineers!"

It's a hard fact of life that some specific individuals play more important roles in successful projects than others.

It's Ilya who conceived of the vision for ChatGPT. Sam is a sales and fundraising guy. He was endorsed by Thiel and Musk.

While raising money is certainly important, let's not confuse that for product vision. There are enough guys that can do what Sam does.

If Susan Fowler's book is accurate, Uber under TK was riddled with toxic management and incompetent HR. Yet you will hear people on Twitter reminisce of TK era Uber as the golden period and many would love him back

And if those primary engineers get sucked out of OpenAI, OpenAI won't be able to compete.

OpenAI is a different animal.

SamAltman has the cache to pull out those engineers. Particularly because Ilya's vision doesn't include lucrative stock options.