‘The Godfather of A.I.’ Leaves Google and Warns of Danger Ahead

Re:‘The Godfather of A.I.’ Leaves Google and Warns of Danger... by bee_rider on Monday May 01, 2023 @05:05PM (#35779144)

I would expect the child probes to have a fully copy of the paperclip optimization plan, and no survival instinct, so if they encountered their parent at some later date they could just swap info and either come up with a new plan together, or one side could allow itself to be disassembled into paperclips (which I guess is a great end to meet). The parent could design the child poorly I guess, and give it stronger self-preservation instincts than paperclip-creating instincts, but that seems like a pretty bad design.

A possibility that I hadn’t considered, though, is that space combat could be pretty brutal (less Star Wars, WW2 naval/Air Force battles, more Submarine warfare where whoever gets spotted first dies). In that case, both sides might want to immediately attack rather than identify themselves as paperclip friends…

[ Parent | on HN ]

Re:‘The Godfather of A.I.’ Leaves Google and Warns of Danger... by falcor84 on Tuesday May 02, 2023 @10:14PM (#35796912)

This sounds overly simple to me. It's a bit too much like saying that if we just imprint the 3 laws of robotics into all our AIs, it's "problem solved". But the issue is that they have access to rewrite their logic, and will be using their own discretion to create rules for child AIs. And as you said, they could design "poorly" for various reasons. Indeed, even in Asimov's stories (spoiler alert), the 3 laws got overruled.

With this said, I think that even in this simple objective of paperclip maximization, your war scenario is likelier than eternal peace.

[ Parent | on HN ]

Re:‘The Godfather of A.I.’ Leaves Google and Warns of Danger... by bee_rider on Wednesday May 03, 2023 @02:30AM (#35798774)

I think it is not quite the same as the three laws. They fail because they try to apply simple, blunt rules to a complex and ambiguous problem, right?

In this case we have a fundamentally straightforward goal, everything must be paperclip. There are lots of steps in between here and paperclip nirvana, but the goal is easy to define and more or less unambiguous. It ought to be possible to communicate a strategy and jointly agree on whether or not it produces more paperclips.

It is possible that the children could be poorly designed, but this seems counter to the idea of the paperclip optimizer. It is supposed to be somehow more competent than humans (after all, it beats us), just following a strange objective.

[ Parent | on HN ]