It isn't hard to imagine that a chatbot that seems to know a lot, and which can easily be convinced to produce text about any arbitrary subject with no grounding in reality, and which is prone to just making up plausible sounding text which is written in an authoritative tone, could be something that could easily trigger such psychotic episodes.
And it doesn't seem improbable that the interactivity of it, the fact that it responds to what is going on in someone's mind, could make it even more prone to triggering certain types of psychosis more easily than traditional unidirectional media like writing, TV, or radio.
Now, that's all supposition. For now, we just have a few anecdotes, not a rigorous study. But I definitely think it is worth looking into whether chatbots are more likely to trigger psychotic episodes, and if there are any safety measures that could be put in place to avoid that.
It's possible that someone already on the verge of a break or otherwise in a fragile state of mind asking for help with their theories could end up with an LLM telling them how incredibly groundbreaking their insights are, perhaps pushing them quicker, deeper more unmoored in the direction they were already headed.
Your example might sound ridiculuous, but this is actually happening all the time. People might not literally jump off a roof, but they do blow themselves up with bombs, go on killing sprees or commit suicide because someone told them so.
Sure, but saying that every religious teacher is fine would be equally wrong. We have a sense that charismatic figures have a corresponding responsibility to their followers (even if they never asked for those followers!) and that broader society should be concerned about what's going on at religious gatherings, especially novel ones, that there's a real risk they can shade into dangerous cults.
If a person had convinced somebody to kill themselves then they would be, rightly, prosecuted for it.
Yes. Unabashedly and without the slightest hesitation.
If your conspiratorial mind has the power to override the most obvious survival instinct, you aren't long for this world.
The paying dividends bit is exactly what society can't afford to cut the psychopaths at the top loose with. The only counterbalance against that rank psychopathy is the fact they'd have to have moral buy in from the people doing the work to implement, giving a de facto check on their power. Remove that element, remove the check.
I agree all around, but/and:
Dividends will be collected by the ones who write such checks; we’ll all pay, as the institutional buy-in has already been priced in.
> Those of us who stand outside the circle of this society’s definition of acceptable women; those of us who have been forged in the crucibles of difference — those of us who are poor, who are lesbians, who are Black, who are older — know that survival is not an academic skill. It is learning how to stand alone, unpopular and sometimes reviled, and how to make common cause with those others identified as outside the structures in order to define and seek a world in which we can all flourish. It is learning how to take our differences and make them strengths. For the master’s tools will never dismantle the master’s house. They may allow us temporarily to beat him at his own game, but they will never enable us to bring about genuine change.
Why even write this? It comes across as so cold. Like mentally ill people just deserve to die since they are destined to die? Incredibly reductive and toxic.
One proxy signal is counting the number of people who start crying about their "good intentions" or searching for thing to blame post facto.
Jurassic Park wasnt about Dinosaurs, it was about the limitations of the 3 inch chimp brain in controlling things that cannot be controlled.
Yudkowski’s point was that we’ve been operating under the assumption that generative AI is aligned by default with humans. Cases like this call that into question.
Even if you hold that you need that genetic predisposition, and that a perfectly sane person can never become "insane" as a result of an experience (not a position I agree with), there is still the very real possibility that many people with the predisposition would never have had this "already insane" condition ever triggered or exposed. Think about the suicide cults that have formed, notably Jonestown. It's easy to consider the mass suicide (and murder of the members' own children) as "insane" behavior. What are the odds that every single person was genetically insane? Comparatively, what are the odds that an extremely persuasive and charismatic personality can rewire a person's reasoning to believe something "insane"?
If you think of ChatGPT as capable of being an automated Jim Jones, I think the danger becomes more clear. These tools are not incapable of being dangerous.
Thinking back on the documentaries about NXIVM, personas like Keith Raniere also seem eerily similar to LLMs in the sense that you can give them any prompt on the spot, and they will start generating long speeches which are essentially bullshit but on the surface seem coherent or even deep.
People don't join cults because of "dangerous information".
The Joe Rogan of AI.
It's lazy, but IMO not for any of the subsequent narrative about media studies. It's lazy because the people were obviously already suffering from psychosis. ChatGPT didn't make them insane. They were already insane. Sane people do not believe ChatGPT when it tells them to jump off a roof! And insane people do insane things literally all the time. That's what it means to be insane.
The idea that ChatGPT convinced someone into being insane is, honestly, insane. Whatever merit one thinks Yudkowski had before this, this feels like a strong signal that he is now a crackpot.