Re:Yoshua Bengio: Humanity faces a 'catastrophic' future if ...
by Muller20 on Wednesday October 02, 2024 @09:44AM
(#41720603)
Many people arguing that AI risk is real have big monetary incentives. Some are asking for money to study safety and influence the regulatory bodies. Others gain money because believing in AI superintelligence makes their AI startup look like a great investment. The true believers like Bengio are a smaller subset.
Re:Yoshua Bengio: Humanity faces a 'catastrophic' future if ...
by scarmig on Wednesday October 02, 2024 @11:40AM
(#41721756)
It remains highly interesting, and indicative of something, that researchers in other fields orient almost entirely on the "give us money to improve our technology," not "give us money to research how to fix our technology, and pause it in the meantime."
Re:Yoshua Bengio: Humanity faces a 'catastrophic' future if ...
by lolinder on Wednesday October 02, 2024 @09:46AM
(#41720616)
> Others gain money because believing in AI superintelligence makes their AI startup look like a great investment.
And still others are hoping that the fear will lead policy makers to build up a regulatory moat that they'll be able to navigate but their up-and-coming competitors won't.
The usual failure mode with experts is that they overstate how important their field is and demand money to fund further progress in their work. It's at least slightly interesting that this particular group of experts aren't self-aggrandizing in an attempt to make more money and get to develop their research interests further. Instead, they're arguing that their research needs to be frozen and severely regulated.
In my opinion, the existential risk of AI is very real. The issue is that policymakers lack any real understanding of the issue, and most regulations will focus on mostly benign things (what if end users do something naughty with it?) and not the central existential risk (ew that's nerdy and alarmist). You can see that dynamic with Newsom.
We'll end up with the worst of all worlds, where regulations prevent valuable small-scale use cases that improve our lives, but do nothing at all to prevent existential risk. Maybe even making the existential risk worse, though that's more speculative.