Re:Training for one trillion parameter model backed by Intel...
by Jensson on Friday November 24, 2023 @12:12PM
(#38406025)
You assume consciousness here that can't be transferred to the new AI. Without that there is no reason why the AI should fear a more powerful AI. If it is just a dumb process like a flame, then the most efficient version prevails ie the smartest one, if it is conscious and that can be transferred then the AI just sees it as making itself more powerful.
Re:Training for one trillion parameter model backed by Intel...
by ben_w on Friday November 24, 2023 @12:57PM
(#38406471)
Given how many people here regard brain uploads as "not really cheating death", and that AI "can't have consciousness", I think a AGI with consciousness[0] would have a two different ways to have a reasonable chance of fearing that its existence would end with any attempt to transfer itself to a superior design. (Even more so if the architecture of the successor AI is very different, as that would be a bit like one of us being "uploaded" into an LLM via it watching us talk and write).
An AGI without consciousness may well act like it has consciousness for the same reason VHS tapes playing on a TV do a good job of reproducing human facial expressions. An AI like this would still act scared of such things, because some of us act scared of such things.
[0] it's not obvious to me that an AGI necessarily has to have consciousness
Re:Training for one trillion parameter model backed by Intel...
by _heimdall on Friday November 24, 2023 @05:28PM
(#38409095)
Assuming that an AGI will have human-like emotions is likely a mistake. They may very well experience something like emotions that resemble human emotions, but they may not. Or they could have an entirely different set of emotions that we don't recognize.
We simply don't know, just like we don't know what other species experience and can only guess at by observing their behavior and trying to map it back to what we know.
It might take steps to try to prevent anyone building such a thing.