To name two Turing award winners: are you asserting that Geoff Hinton and Yoshua Bengio are pushing the AI risk narrative primarily because they're compromised by financial incentives?

Geoff Hinton /left his job/ at Google because he sees the risks as real, so I think that's a tough case to make.

> are you asserting that Geoff Hinton and Yoshua Bengio are pushing the AI risk narrative primarily because they're compromised by financial incentives

Their message is being amplified and distorted by forces that are. That they're both comfortably wealthy from creating the problems they now preach solutions to is also no small matter.

> Geoff Hinton /left his job/ at Google because he sees the risks as real, so I think that's a tough case to make.

He did not leave his job when the bias and harm of the existing I models at Google were doing.

He did not quit his job when Google fired their AI ethics champion rather than change their behaviour

Has he had anything to say about the harm Google's algorithms do?

At least that's what he says. Maybe he got an unspecified offer to be the high priest of a to-be influential 'AI safety' organization. In my opinion, just like with the Patriot Act we are seeing a new world order trying to come into being. Come a decade, the few who bother and can learn about what transpires in the near future, will be shaking their heads at present humanity, asking how could we be so stupid and naive.

I don't like the prevailing cynicism of dismissing the content of the message just because some bad/prejudiced/wrong factions are involved in it. It's not the way I learned to do research and answer questions. It's nevertheless an ad hominem to dismiss an issue just because some people out there are trying to co-opt it. There's a psychological explanation for that cynical behavior: internalized oppression. The focus is put on the morality of the people saying a thing, rather than just looking at the thing and deciding, independent of the existence of those groups, the objective, scientific implications.

If it were a perfect society and people discovered deep neural networks, there would be no excuse to blame bad actors for posing the question of whether AGI is an existential problem or not, and what to do about it. Unlike bad groups of people, the question won't go away. In the real world, it is entirely possible to have to consider multiple issues at once, not at the expense of any issue.

Are there any previous cases you can think of where an industry exaggerated the dangers of their technology in order to increase profits? It seems to me that, if that is what is going on, its the first time this strategy has ever been attempted. I cannot think of a single case where manufacturers of cars, prescription drugs, firearms, chemicals, social media, airplanes, or nuclear power plants have exaggerated the dangers of their technology. In fact its common (perhaps universal) to downplay the dangers.

Perhaps the simplest explanation for why researchers are saying there is a danger is that they genuinely believe it.

They're exaggerating the efficacy of their technology, and are demanding a regulatory moat that currently doesn't exist under the pretext of the danger of that technology being misused. Notice the singular focus on "extinction risk" rather than flaws that the technology actually has right now, like hallucination, training bias, and prompt injection or the looming harm of AI adoption in both putting people out of work while substantially reducing the quality of anything automated with it. Part of it is marketing and part of it is ego on the part of people who grew up on fictional narratives about AI destroying the world and want to feel like they wield the power to make it do that, but whether any individual's belief is sincere or not, the overall push is a product of the massive incentives everyone involved in AI has to promote their product and entrench their market position.

You're saying that AI researchers are not immune to the power of incentives. I'm saying that there is no evidence that this sort of incentive has ever caused this sort of behavior before.

The Catholic church selling indulgences: hype the potential problems, profit from the panic.

Yudkowsky and many others have been working on this problem for decades, way before there was any financial incentive to speak of.

Yeah and they finally get their 15 minutes. Obviously they'll take it...