The actions of other organisations are not in the scope of the board's mission. The actions of the company the board controls are in that scope.

"The board’s mission is to ensure that the company creates artificial intelligence that “benefits all of humanity,”"

We can't control other's actions, but we can control our own. If we feel that our actions are diverging from our own sense of what we ought to be doing we can change our actions, regardless of how others are behaving.

That is absolutely true. But you do operate on a larger stage and sticking your head in the sand and pretending the rest of the world doesn't exist is also not going to work.

Altman appears to me to be reckless, ms. Toner appears to be naive. Both should not be near this kind of power.

>But you do operate on a larger stage and sticking your head in the sand and pretending the rest of the world doesn't exist is also not going to work.

I'm sure that's how tobacco and oil executives justify their paycheck: "If I wasn't doing it, someone else would". Probably lots of criminals justify their crimes using similar reasoning too.

Ultimately it's not quite true. If you quit your job at that tobacco or oil company, it's going to take a while to find them a replacement for you. That replacement might not be quite as good at the job (why were you picked for the job in the first place? Probably because you were the strongest candidate.) You aren't going to totally halt EvilCorp by quitting, but you're going to give it a speed bump.

In the case of OpenAI, we have to consider what organization is likely to replace them if they throw in the towel. If it's a more cautious org like Anthropic, OpenAI throwing in the towel could actually be a good thing from the point of view of their charter: https://openai.com/charter

I can see a number of points in time where that goal could have been achieved much better than in the way it happened. For instance: instead of a single deal with a large tech company it should have been a financial deal shared between a syndicate.

That would have been a lot safer than to give MS the keys to the kingdom.

But she was on the board when that happened and presumably agreed to the deal or she could have resigned.

And it's not quite the tobacco company we're talking about here. I'm more in line to compare it to the Manhattan project.

That's not obvious to me. Suppose OpenAI deliberately partners with a number of different infra providers. That creates a bunch of drag because they have to create a compatibility layer for staying agnostic to the underlying infra. And the outcome in a situation like this would probably be fairly similar: Since big tech companies are focused on making a buck, they would probably pressure the board to do whatever they think will be most profitable, just like Microsoft.

Additionally, the role of the board is to supervise the CEO, not make strategic decisions, no? If the MS partnership was a mistake, Sam is the primary person to blame here.

I'm seeing a lot of "Monday morning quarterback" in this thread

> I'm seeing a lot of "Monday morning quarterback" in this thread

That's fine but you are not forced to participate.

> That's not obvious to me. Suppose OpenAI deliberately partners with a number of different infra providers. That creates a bunch of drag because they have to create a compatibility layer for staying agnostic to the underlying infra. And the outcome in a situation like this would probably be fairly similar: Since big tech companies are focused on making a buck, they would probably pressure the board to do whatever they think will be most profitable, just like Microsoft.

So, either you are 'OpenAI' and that means open to all or you are Microsoft AI, licensing your tech to one of the least ethical and most wealthy companies in the world. If the board didn't have anything to say about that at the time and if they didn't resign en masse you have to assume that they are ok with it.

> Additionally, the role of the board is to supervise the CEO, not make strategic decisions, no? If the MS partnership was a mistake, Sam is the primary person to blame here.

So are they or are they not there to supervise the CEO?

In my view a board exists to protect the interests of the stakeholders guided by a charter. OpenAI sees 'humanity' (whatever that means to them) as their stakeholders and that trumps each and every other party involved. But it is unclear whether their actions really benefit those stakeholders or if it is just people being people.

So far I'm seeing a lot of the latter and only little bits of the former.

I would not say that it is fair comparison in this case. People do the best with the control they have.

In this case, if your company fails to provide for greater good, then it is best to destroy it. And it also shows some example to others. Whatever others do, you have no power. I does not matter if you pretent or not.

Comparison works only if you could do something about it.

Yes, but then maybe you should realize that you are out of your depth and abstain. Because clear as day this kind of power is actively being sought by many operators and if OpenAI was 'the good guys' then any chance of the good guys getting a head start has just been blown out of the water.

If she agreed with the Microsoft deal but not with what followed that too would be hopelessly naive. Microsoft + ethics?

There are no good guys only good deeds

> If she agreed with the Microsoft deal but not with what followed that too would be hopelessly naive. Microsoft + ethics?

I guess the main problem here is that nobody knows properly what this deal contains and how it was made. I believe CEO has power to do such deal alone?

That depends on the mandate. He would bind the company but it is possible that his mandate specifically forbids deals that include either IP or exclusivity.

But what is interesting is that until MS got involved OpenAI was struggling for funding and talent, and afterwards it took off like a rocket.

I think the talent was already there but the power of Azure servers was the critical booster. We don’t even know how much Microsoft has actually paid instead of just counting Azure credits.

Yes, Nadella has a point: without MS OpenAI wouldn't be where it is today, but the reverse is also true: without OpenAI MS wouldn't be where it is today.

arguably no one in openai should. the ability to develop software has nothing to do with the ability to make decisions for the world at large.

That may well be true. But that invites musings about nationalization. Which I'm sure more than one person has been thinking about already.

This is an all but undeclared arms race.

Indeed. The only way to retain control of the leading ship in the race is to keep it together as you steer it. If the ship disintegrates, then you're no longer in control of the leading ship, and someone else will win the race.

The charter states that OpenAI should allow another company to win the race under certain circumstances:

>We are concerned about late-stage AGI development becoming a competitive race without time for adequate safety precautions. Therefore, if a value-aligned, safety-conscious project comes close to building AGI before we do, we commit to stop competing with and start assisting this project. We will work out specifics in case-by-case agreements, but a typical triggering condition might be “a better-than-even chance of success in the next two years.”

https://openai.com/charter

That's why Sam pressuring Helen to shut up about Anthropic is so concerning.

> we commit to stop competing with and start assisting this project

That bit I never understood. It doesn't even have a qualification as to who is stewarding the project as if any AI should be assisted no matter who ends up owning it. The whole thing is just so utterly disconnected from the real world.

If you believe that aligned AGI will create a post-scarcity society, there's no reason for its creator to hoard the benefits for themselves. The main thing is to ensure that post-scarcity society actually comes into being.

I don't believe that.

What I believe is that the first party to create an AGI will likely end up being nationalized and that that AI will be weaponized the next day to the exclusion of the rest of the world.

That's fine, but the OpenAI founders (including Sam Altman![1]) explicitly had in mind "AI kills everyone" as one of the main problems they were attempting to tackle when they started the organization. (Obviously starting OpenAI as a solution to this problem was a terrible mistake, but given that they did so, having a board charged with pulling the plug is better than not having that.)

[1] https://blog.samaltman.com/machine-intelligence-part-1

That whole charter is bullshit. You don't invite Microsoft to the party if you care about ethics and you don't disable your fail-safe by giving them full access to all of your stuff.

Hence my position that the charter was a fig leaf designed to throw regulators off.

Elsewhere in this thread you stated: "if OpenAI was 'the good guys' then any chance of the good guys getting a head start has just been blown out of the water"

So basically we have two cases here:

1. The charter is a deceptive fig leaf, OpenAI aren't the good guys. In which case blowing up OpenAI looks reasonable

2. The charter is not a fig leaf, Helen is doing the right thing by upholding it

I'd say upholding the charter is the right thing in intermediate situations as well. Basically, OpenAI are the good guys, in part, to the degree to which the board (and other stakeholders) actually uphold the charter instead of being purely profit-driven.

There's a third case:

It's #1 to some, and #2 to others.

there is no such thing as general post-scarcity though. there is post-scarcity of certain needs but not general post-scarcity. So it's creator would still have reason to hoard certain benefits for themselves while distributing some for all.

Once you get past a certain level of wealth, the esteem of your fellow humans matters more than acquiring even more goods and services.

That's statistically speaking not very well supported. The most common element of those that are past a certain level of wealth is that they want more wealth, more power or both.

It's generally meant that material scarcity becomes secondary to social wellbeing. Ie, material needs are so easy to take care of that the dynamics of social interaction undergo a fundamental shift, as they currently are strongly tied to material conditions.

Did you miss the part where it says "value-aligned"? One's that aren't value-aligned are not subject to that clause.