Re:The Reluctant Prophet of Effective Altruism
by kalkin on Tuesday August 09, 2022 @10:22PM
(#32407219)
Depends on the question. I agree that the argument for a random person giving money to try to reduce AI risk seems dubious, and touched on this in another comment - https://news.ycombinator.com/item?id=32394429 - although I'm not up to speed on the "longtermism" debate as such.
But I think the argument for a policymaker or ML researcher to be worrying about this is at least better than Pascal's Wager (and remains so whether or not you buy any particular whiff-of-Pascal claims about simulations, "acausal trade", etc).
Re:The Reluctant Prophet of Effective Altruism
by notahacker on Wednesday August 10, 2022 @06:46AM
(#32409973)
Agree on both points. (There's a perfectly good reason for EAs with a computer science skillset to want to work in AI with a focus on safety; it's a well compensated industry where they can probably solve important real world problems more reliably than AI researchers less concerned about the ethical implications of what they do even without any sort of singularity, it just the movement seems to have invested a lot more in talking about longtermism than anything else lately, and Yudowksy's always had some very... individual lines of reasoning)
Yudkowsky's personal estimates of the probability of superhuman AI and the probability of his own research institution fixing it may have been more confident than most people's but he and his followers are no stranger to presenting Pascals' wager arguments to people who doubt approaches are feasible (even more so when it comes to stuff he isn't working on like cryonics), along with more explicitly Pascal-esque stuff like intertemporal bargaining with future AIs, simulations and infinite rewards/punishments...