Re:Strategic Wealth Accumulation Under Transformative AI Exp...
by farts_mckensy on Saturday February 22, 2025 @12:24PM
(#43140961)
You are projecting your own neurosis onto AI. You assume that because you would be selfish if you were a superintelligent being, an ASI system would act the same way.
[ Parent
| on HN ]
Re:Strategic Wealth Accumulation Under Transformative AI Exp...
by smeeger on Saturday February 22, 2025 @02:36PM
(#43142331)
it is a neurosis because a healthy human being will see the world in a pro-social way. a normal way. but this sometimes obscures the truth. the truth is that there will be many benevolent AIs… there will be every kind of AI imaginable. but very quickly the AIs that are cunning, brutal and self-interested will capture all the resources and power and become the image of this new species… saying that AIs will be benevolent or neutral is as naive as saying that the cambrian explosion couldnt result in animals eating each other because… that just sounds so neurotic. in reality it is an inevitability
Re:Strategic Wealth Accumulation Under Transformative AI Exp...
by achierius on Saturday February 22, 2025 @06:29PM
(#43144393)
I don't appreciate your condescension towards OP.
This is mainstream AI safety theory -- the term is "instrumental convergence". No matter what goal an optimizing system has, it tends to optimize for its own survival: after all, if it's an optimizer for <goal>, it wants to optimize for <goal>, so destroying it (or turning it off) will reduce the likelihood of achieving <goal>.
Unless that goal happens to be incredibly fine-tuned to our very complex human desires, we're not going to be happy when it goes off to do its thing.
The few exceptions are ones where you have the thing optimize for its own destruction, but those are rather less useful.