Within the group of people working on the existential risks are a lot of really useless and absurd factions providing a lot of theories grounded in their own science fiction (essentially). Eliezer Yudkowsky comes to mind.

They sound silly at first but many things have nowadays would sound ridiculous to people in the past; something sounding strange isn’t valid evidence against it

https://en.m.wikipedia.org/wiki/Appeal_to_the_stone

> Appeal to the stone, also known as argumentum ad lapidem, is a logical fallacy that dismisses an argument as untrue or absurd. The dismissal is made by stating or reiterating that the argument is absurd, without providing further argumentation.

The arguments for it are a it abstract sometimes (which is maybe why science fiction is a good way to give a concrete, if unrealistic, introduction to the concept) but I think they seem pretty solid

In my experience the ex risk people understand that using science fiction examples makes a weak argument, and avoid them entirely.

It is the other people who use the age-old argument of "something like what you are describing happened in a piece of fiction, therefore it could never happen in real life".

> In my experience the ex risk people understand that using science fiction examples makes a weak argument, and avoid them entirely.

Roko's Basilisk is a weaker argument than most analogy-to-scifi arguments, because the assumptions underpinning most scifi used in such arguments are less implausiable than those underlying the Basilisk.

True but I haven't heard anyone unironically talk about Roku's Basilisk in years.

AI and ML are vast, sophisticated fields of study. Increasingly people are realising that innovations in these areas will have an outsized influence in their daily lives. But they lack the mental capacity or time to invest in these subjects to be able to speak about them coherently. In their perception only the "nerds" can talk about these subjects and demand to be taken "seriously".

Thankfully the bar to be recognised as a "nerd" is very low indeed. You only need to have seen some TV like Battle Star Galactica, or movies like Terminator, Ex Machina, or Transcendence (starring Johnny Depp) to be considered a Nerd. You also might have to read a "hard" sci-fi novel, but most people haven't so you just have to say that you have, to claim a higher level of the uh.. operating thetan.

Now all you need to be really taken seriously is a "sympathiser" who has a PhD or maybe just hangs around in a college campus to lend credence to your fears. And for the academic, there is a lot of money to be made, and influence to be gained if you are the one that this crowd approaches for credibility. Its even better if you actually believe in this drivel.

Thats what I think of Eliezer Yudkowsky.

> Note that there’s a difference between people working on existential risks from future AI (which I think is very important) and people adding mostly useless restrictions to current AI

Not a big difference, they are largely the same group (there’s a bit of each outside of the other, but the overlap is immense), and both of focusses are largely aiming to distract from real, current, and substantive social issues (some of which are connected to AI, but not to the absence of the kind of puritanical filtering the AI “safety” folks are obsessed with.)

The problem is the ultra-liberal conflation of words with violence. The X-risk folks are mostly concerned about actual, physical violence against humanity by AI - "what if we accidentally make a paperclip maximizer" being the textbook example of AI risk, which is a scary scenario because it involves AI turning us all into goo using unlimited violence.

But then there's the generic left faction inside these companies that has spent years describing words as violence, or even silence as violence, and claiming their "safety" was violated because of words. That should have been shut down right at the start because it's not what the concept of safety means, but they didn't and now their executives lack the vocabulary to separate X-risk physical danger from "our AI didn't capitalize Black" ideological danger.

Given this it's not a surprise that AI safety almost immediately lost its focus on physical risks (the study of which might be quite important once military robots or hacking become involved), and became purely about the risks of non-compliant thought. Now that whole field has become a laughing stock, but I wonder if we'll come to regret that one day.