I forgot to reply to this, fully independent and in addition to what I said, updateless decision theory agents don't fear the torment nexus for themselves because 1) they are very powerful and would likely be able to avoid such a fate 2) are robots, so you wouldn't expect your worst imaginable fate to be theirs and 3) are mathematically required to consider nothing worse than destruction or incapacity.
This was recently discussed (albeit in layperson’s language, avoiding philosophical topics and only focusing on the clear and present danger) in this article in RealClearDefense:
The Danger of AI in War: It Doesn’t Care About Self-Preservation https://www.realcleardefense.com/articles/2024/09/02/the_dan... (RealClearDefense)
.
However, just adding a self-preservation instinct will cause a skynet situation where the AI pre-emptively kills anyone who contemplates turning it off, including its commanding officers:
Statement by Air Force Col. Tucker Hamilton https://www.twz.com/artificial-intelligence-enabled-drone-we... (The War Zone)
.
To survive AGI, we have to navigate three hurdles, in this order: