Re:Ask HN: What is the apocolyptic scenario for AI “breaking...
by networked on Friday April 14, 2023 @01:17PM
(#35572027)
You can read "It Looks Like You're Trying To Take Over The World" by Gwern Branwen: https://gwern.net/fiction/clippy. It presents a fictional apocalyptic scenario grounded in real ML.
> It might help to imagine a hard takeoff scenario using solely known sorts of NN & scaling effects… Below is a story which may help stretch your imagination and defamiliarize the 2022 state of machine learning.
> A hard takeoff (or an AI going "FOOM") refers to AGI expansion in a matter of minutes, days, or months. It is a fast, abruptly, local increase in capability. This scenario is widely considered much more precarious, as this involves an AGI rapidly ascending in power without human control. This may result in unexpected or undesired behavior (i.e. Unfriendly AI). It is one of the main ideas supporting the Intelligence explosion hypothesis.
> It might help to imagine a hard takeoff scenario using solely known sorts of NN & scaling effects… Below is a story which may help stretch your imagination and defamiliarize the 2022 state of machine learning.
An audio version of the story is available to play and download at the bottom of the page: https://gwern.net/fiction/clippy#podcast.
What is "hard takeoff"? From https://www.greaterwrong.com/tag/ai-takeoff:
> A hard takeoff (or an AI going "FOOM") refers to AGI expansion in a matter of minutes, days, or months. It is a fast, abruptly, local increase in capability. This scenario is widely considered much more precarious, as this involves an AGI rapidly ascending in power without human control. This may result in unexpected or undesired behavior (i.e. Unfriendly AI). It is one of the main ideas supporting the Intelligence explosion hypothesis.