Re:Ask HN: What is humanity building towards?
by endominus on Tuesday December 21, 2021 @04:50PM
(#29642565)
If the technology is infinitely advanced, why do we believe it would obey us? Would you obey the commands/directives of protozoic slop, or even a mentally disabled early primate? We would depend instead on the AI deriving a morality that benefits us on its own. But why would the morality that an AGI derives, based on nonhuman thinking, in a more complex and alien worldview than we can possibly comprehend, align with our own extremely narrow, frequently inconsistent values? It's not a matter of cognitive biases or misunderstanding. The idea that a sufficiently intelligent AGI would just decide to enslave itself for humanity's benefit is extremely common and extremely odd to me.
[ Parent
| on HN ]
Re:Ask HN: What is humanity building towards?
by netsec_burn on Tuesday December 21, 2021 @05:34PM
(#29642936)
I agree that regardless of how it is created, it is unlikely to obey or be enslaved. I also agree that the same selective pressures that caused altruism/symbiotic relationships to be an emergent property in animals wouldn't exist for an AGI. That's a lot to think about, thanks for explaining it to me.