I agree with you, as “smart” as these people at AI companies are, somehow they keep missing the fact that it’s impossible (unless you kill the majority of people) to align on this kind of thing

I can't imagine a world where this happens.

How is it possible to have something that is autonomous, intelligent and completely aligned?

There is a simple way to fix the alignment problem ,don't build systems that require aligning. I'm not really talking about regulation, I'm just talking about using common sense and wisdom.

The reality is we all would love an AI that improves on its own. Look at all those chatgpt examples that were shared because it got something wrong. If only we could make gpt learn then it would only get it wrong once that would make it so much more useful.

And therein is the problem we all want this. But as soon as we allow the AI to learn on its own we have introduced the alignment problem. And of course on day 1 nobody will see gpt correcting itself on simple questions as an issue. But on day 365 after it’s been learning at an exponential rate and we’ll have all realized that it’s too late because it’s learned faster than we have.

Has anyone thought about what it would actually take to make gpt learn at an exponential rate? GPT4 from 3 was already starting to see diminishing returns. GPT3 cost $4.6 million dollars to train. 4 cost $100 million, an order of magnitude more, and required an enormous amount of data. All these training algorithms are O(n^2). Do we actually expect these things to grow exponentially or is it more realistic to expect we're going to spend a lot of compute and treasure only to see disappointing results?

We have entire fields that have been trying to solve the alignment problem for autonomous + intelligent agents for centuries: economic and political science.

Feels a little bit different though ?

...also, not sure if this is the point you're trying to make, but nothing is aligned with human interests really, look at what economics and the governmnet has done with climate change?

"Trying": The Best We Can Do!

Yeah, and so far, they've failed. So that's not really encouraging.

Exactly. Who is “we”? What are “our” goals? There are (nearly?) no universal human values.

AI doesn’t have ethics, it doesn’t have values. It might imitate those in some ways, but it doesn’t have them.

What this article longs for is a God it can control. Good luck with that.

It's actually pretty simple. Like many "innovations" in the past several years, alignment means profitable for founders and investors, and whatever enables a small minority to reach unthinkable levels of productivity, wealth and control so everybody else can serve them as waiters, cleaners and prostitutes.

Then they will fly their AI plane (the pilot got fired too, now mowing the lawn) to an ultra-wealthy paradise to discuss with the other billionaires how to "reduce inequality". The proposed solution will be even more "innovations"... THIS time around the next wave of innovation will actually be redistributive, so they say, but in fact it will be rinse and repeat.

We haven’t managed to align capitalism with the continued survival of the human race or even basic equity. And that’s running on a minimal distributed system of rules and meat bags as CPU

It’s preposterous to even hint we could solve AI alignment.

Never interrupt a Silicon Valley billionaire who is busy making a fool of himself, twisting logic in complex ways to explain why he should be allowed to run the new AI bubble, to add even more infinite money to his already enormous pool of infinite money...

On that second point... haven't we kind of solved this? Or rather there's a really obvious parallel. We've got an extremely powerful thing that can do whatever we want it to do, and now all we need to do is decide what to ask it to do and make sure it does what we want. Where else do we see that? Well that's government. The US is a massively powerful country, it's got hundreds of thousands of soldiers, billions of dollars at it's disposable, it can shoot individual humans into the sun if it wants to -and it's intelligent too! So how do we control it? A set of institutions with checks and balances elected by and accountable to the public.

But stable/self-constrained governments require consensus to operate. They doesn’t just ask politely and prefer it. Checks and balances are preserved not because all the parties are interested in preserving them overall, but because they each want their own power not to be impeded. The systems that can’t achieve this balancing act fall over after a while, and this happens very frequently (99.999% of governments in history?).

Nuclear MAD produces a similar equilibrium but this is stable due to specific characteristics of nuclear weapons and their delivery systems (expensive to produce; almost impossible to produce, test, or use secretly; impossible to reliably intercept). We shouldn’t bet on AI development reaching the same equilibrium.