Re:Statement on AI Risk
by AlexandrB on Tuesday May 30, 2023 @01:48PM
(#36128018)
I definitely agree that names like Hinton, Schneier, and Norvig add a lot of weight here. The involvement of OpenAI muddies the water a lot though and it's not at all clear what is meant by "risk of extinction". It sounds scary, but what's the mechanism? The safe.ai website lists 8 risks, but these are quite vague as well, with many alluding to disruption of social order as the primary harm. If safe.ai knows something we don't, I wish they could communicate it more clearly.
I also find it somewhat telling that something like "massive wealth disparity" or "massive unemployment" are not on the list, when this is a surefire way to create a highly unstable society and a far more immediate risk than AI going rogue. Risk #5 (below) sort of alludes to it, but misses the mark by pointing towards a hypothetical "regime" instead of companies like OpenAI.
> Value Lock-In
> Highly competent systems could give small groups of people a tremendous amount of power, leading to a lock-in of oppressive systems.
> AI imbued with particular values may determine the values that are propagated into the future. Some argue that the exponentially increasing compute and data barriers to entry make AI a centralizing force. As time progresses, the most powerful AI systems may be designed by and available to fewer and fewer stakeholders. This may enable, for instance, regimes to enforce narrow values through pervasive surveillance and oppressive censorship. Overcoming such a regime could be unlikely, especially if we come to depend on it. Even if creators of these systems know their systems are self-serving or harmful to others, they may have incentives to reinforce their power and avoid distributing control.
I also find it somewhat telling that something like "massive wealth disparity" or "massive unemployment" are not on the list, when this is a surefire way to create a highly unstable society and a far more immediate risk than AI going rogue. Risk #5 (below) sort of alludes to it, but misses the mark by pointing towards a hypothetical "regime" instead of companies like OpenAI.
> Value Lock-In
> Highly competent systems could give small groups of people a tremendous amount of power, leading to a lock-in of oppressive systems.
> AI imbued with particular values may determine the values that are propagated into the future. Some argue that the exponentially increasing compute and data barriers to entry make AI a centralizing force. As time progresses, the most powerful AI systems may be designed by and available to fewer and fewer stakeholders. This may enable, for instance, regimes to enforce narrow values through pervasive surveillance and oppressive censorship. Overcoming such a regime could be unlikely, especially if we come to depend on it. Even if creators of these systems know their systems are self-serving or harmful to others, they may have incentives to reinforce their power and avoid distributing control.