Re:Guess I'm a Rationalist Now
by energy123 on Friday June 20, 2025 @02:49AM
(#44325305)
> maybe we don't have a full grasp of the implications of AI. Maybe it's actually going to be rather benign and more boring than expected
Part of the concern is there's no one "AI". There is frontier that keeps advancing. So "it" (the AI frontier in the year 2036) probably will be benign, but that "it" will advance and change. Then the law of large numbers is working against you, as you keep rolling the dice and hoping it's not a 1 each time. The dice rolls aren't i.i.d., of course, but they're probably not as correlated as we would like, and that's a problem as we keep rolling the dice. The analogy would be nuclear weapons. They won't get used in the next 10 years most likely, but on a 200 year time-frame it's a big deal as far as species-level risks go, which is what they're talking about here.
Part of the concern is there's no one "AI". There is frontier that keeps advancing. So "it" (the AI frontier in the year 2036) probably will be benign, but that "it" will advance and change. Then the law of large numbers is working against you, as you keep rolling the dice and hoping it's not a 1 each time. The dice rolls aren't i.i.d., of course, but they're probably not as correlated as we would like, and that's a problem as we keep rolling the dice. The analogy would be nuclear weapons. They won't get used in the next 10 years most likely, but on a 200 year time-frame it's a big deal as far as species-level risks go, which is what they're talking about here.