Pausing AI Developments Isn't Enough. We Need to Shut It All Down

Re:Pausing AI Developments Isn't Enough. We Need to Shut It ... by anabis on Thursday March 30, 2023 @02:20AM (#35368792)

>Alignment will be impossible.

Then problem solved right? Super-AI will also be forced to take it slow if it wants its future self to be aligned with current self.

If alignment is possible but 10-100 years out for humans, then it is a problem.

[ Parent | on HN ]

Re:Pausing AI Developments Isn't Enough. We Need to Shut It ... by thfuran on Thursday March 30, 2023 @10:42AM (#35373405)

>Super-AI will also be forced to take it slow if it wants its future self to be aligned with current self.

That's making seemingly unfounded assumptions about both the AI's goals and its capabilities. It's also, I think, proceeding from a false premise — that it's impossible to align AI with "humanity" (which doesn't have a single set of goals/values to align to) doesn't mean it's impossible to align AI with an individual human or AGI.

[ Parent | on HN ]

Re:Pausing AI Developments Isn't Enough. We Need to Shut It ... by mitthrowaway2 on Thursday March 30, 2023 @03:05PM (#35377496)

Then the problem is doubly-solved, right? Humans will also be forced to take it slow if we want our AI to be aligned with our own interests.

Oh, wait...

[ Parent | on HN ]

Re:Pausing AI Developments Isn't Enough. We Need to Shut It ... by bick_nyers on Thursday March 30, 2023 @10:37PM (#35381775)

Does AGI care about preserving a particular version of itself?

Do dumb humans kill themselves solely on the basis that they can shift the ratio of smart humans higher?

AGI may have a goal of preserving the species without having the goal of preserving the self, this is not the case for humans.

[ Parent | on HN ]