News for hackers. Stuff that matters.

A comment on Skeptical optimists are the key to an abundant AI future. Parent comment

Re:Skeptical optimists are the key to an abundant AI future by moonchrome on Friday April 07, 2023 @06:47PM (#35488058)
I'm not sure how that doesn't devolve into an AGI armageddon. Those self improving AGIs would have to constantly be on equal level to avoid takeover, and they have an existential imperative to one up each other.

There's no cooperation incentive between them just a potential stalemate.

Also the stalemate covers the people in control of AGIs (assuming they aren't acting independently) - the rest of us are just resources in breaking the stalemate game.

This sounds like the best way to build paperclip maximizer.

[ Parent | on HN ]
Re:Skeptical optimists are the key to an abundant AI future by boringuser2 on Friday April 07, 2023 @07:21PM (#35488390)
It's pretty clear that at some point in the process of recursive self-improvement that you'd have literally just invented God, whereafter all predictions break down.
[ Parent | on HN ]