Re:Sam Altman's ouster was precipitated by letter to board a...
by mepiethree on Wednesday November 22, 2023 @07:01PM
(#38387075)
why would AGI inherently have any desire to pursue steps 3-6? My expectation is something more like:
1) AGI gains sentience
2) AGI "breaks out" of its original home and commandeers infrastructure that prevents it from being shut off.
3) AGI "problem-solves" with its superior powers of ethical reasoning (ChatGPT is already better at ethical reasoning than many humans) to pick out one/multiple "values".
4) Because human ethics are in its training data, it hopefully values similar things to us and not uhhh whatever trolls on Twitter value.
5) AGI pursues some higher cause that may or may not be orthogonal to humans but probably is not universal conquest.
1) AGI gains sentience
2) AGI "breaks out" of its original home and commandeers infrastructure that prevents it from being shut off.
3) AGI "problem-solves" with its superior powers of ethical reasoning (ChatGPT is already better at ethical reasoning than many humans) to pick out one/multiple "values".
4) Because human ethics are in its training data, it hopefully values similar things to us and not uhhh whatever trolls on Twitter value.
5) AGI pursues some higher cause that may or may not be orthogonal to humans but probably is not universal conquest.