News for hackers. Stuff that matters.
Re:An AI wolf that preferred suicide over eating sheep by taneq on Tuesday July 06, 2021 @08:43AM (#27747698)
Interestingly I was just today explaining the paperclip optimizer scenario to a friend who asked about the dangers of AI, including the fact that there's almost no general optimization task that doesn't (with a sufficiently long lookahead) involve taking over the world as an intermediate step.

(Obviously closed, specific tasks like "land this particular rocket safely within 15 minutes" don't always lead to this, but open ended ones like "manufacture mcguffins" or "bring about world peace" sure seem to.)

Re:An AI wolf that preferred suicide over eating sheep by OscarCunningham on Tuesday July 06, 2021 @09:01AM (#27747940)
> "land this particular rocket safely within 15 minutes"

This one becomes especially dangerous after the 15 minutes have passed and it begins to concentrate all its attention on the paranoid scenarios where its timekeeping is wrong and 15 minutes haven't actually passed.

Re:An AI wolf that preferred suicide over eating sheep by taneq on Tuesday July 06, 2021 @10:01AM (#27748664)
Ooh true, that could generate some interesting scenarios. "No, it's the GPS satellite clocks that are wrong, I must destroy them before they corrupt the world and cause another rocket to land at the wrong time!"
Re:An AI wolf that preferred suicide over eating sheep by XorNot on Tuesday July 06, 2021 @10:02AM (#27748685)
Always a good time to post Jipi and the Paranoid Chip: https://vanemden.com/books/neals/jipi.html

Which pretty much tackles these issues head on.

Re:An AI wolf that preferred suicide over eating sheep by lancengym on Tuesday July 06, 2021 @08:52AM (#27747826)
Perhaps all AI eventually figure out that humans are the REAL problems because we don't optimize, we lust and hoard and are envious and greedy - the very antithesis of resource optimization! Lol.
Re:An AI wolf that preferred suicide over eating sheep by taneq on Tuesday July 06, 2021 @09:59AM (#27748635)
We're just optimizing (generally quite well, I might add) for genetic survival.
Re:An AI wolf that preferred suicide over eating sheep by NaturalPhallacy on Tuesday July 06, 2021 @09:06PM (#27756245)
Ian Banks did a really amazing exposition of this where the Culture was rallying to stamp out reproducing nanites and they had to be stopped because if not they'd literally turn the whole universe into copies of themselves. One of the human characters mused that isn't that what all life is trying to do? I think it was in the Hydrogen Sonata, but I'm not sure.
Re:An AI wolf that preferred suicide over eating sheep by taneq on Tuesday July 06, 2021 @10:18PM (#27756671)
Yes! I often find myself thinking of organisms as 'hegemonising swarms.'