The root cause here is the parents. It's visible behind those screenshots. The child clearly didn't trust their parents and didn't feel they cared, listened or tried to understand the problems the teen was going through. This is clearly a parenting failure.
There's far worse content out there than these mild messages that teenagers will be in contact with, starting with 4chan, to various competitive video games to all other sorts of weird things.
This is a cop out for parenting failure, where parents are looking to play victims, since they can't take responsibility for their failures and AI seems like something that could make them feel better.
At 17, it's humiliating to have phone taken away from you in such manner, then your parents going through your phone to find those text messages in the first place. Then making a lawsuit out of this portraying all the intimate details to the public. These parents seem to have 0 care for their child.
0.5% is a really high fraction for fucking up to the point of encouraging kids to murder!
Now put that in a kid’s show script and re-evaluate.
> It's basic literacy to understand that it's not that
You know who needs to be taught basic literacy? Kids!
And look, I’m not saying no kid can handle this. Plenty of parents introduce their kids to drink and adult conversation earlier than is the norm. But we put up guardrails to ensure it doesn’t happen accidentally and get angry at people who fuck with those lines.
We have no evidence of video games causing violence. We have evidence of kids killing themselves after talking to bad chatbots.
GP said: "no evidence of video games causing violence.", which is completely different to what you wrote. I'm sure a lot of violence is committed after lunch.
A chatbot telling a kid to kill themselves and then the kid killing themselves seems like causation [1].
[1] https://www.nytimes.com/2024/10/23/technology/characterai-la...
Big difference is the video game industry had studies to back them up. Where are the data for chatbots? On the benefits? Lack of risk? Is there a single child psychologist in the ranks of these companies?
Video games are also rated, to help parents make age-appropriate decisions. Is Character.ai age gated to any degree?
And while industry research doesn’t equate to bad research, it should be held to a higher standard simply because of the obvious incentives. Would you automatically accept tobacco company research to make strong conclusions about the safety of smoking?
The LLM in terms of edginess is just going to build on your own edginess assuming it is uncensored. It is not going to convince you of something out of nowhere.
Given clear hint that someone is happy with dark humour, LLM should be able to throw some of it out back.
I am just sad the kid has this type of gaslighting parents making him feel that he is in the wrong when he seems more intelligent than they are.
Context is everything; if this had been a conversation with a human it would be hard not to read it as malicious.
(I suspect character.ai was originally conceived precisely because it appeared to be a usecase where LLM unreliability would be okay, the creators not having thought sufficiently carefully about it.)
It's probably true in most cases. Those cases don't cover kids.
I think people are confused by this situation. Our society has restrictions on what you can do to kids. Even if they nominally give consent, they can't actually give consent. Those protections basically don't apply to kids insulting each other on the playground but they apply strongly to adults wandering onto the playground and trying to get kids to do violent things. And I would hope they apply doubly to adult constructing machines that they should know will attempt to get kids to do violent things. And the machine was definitely trying to do that if you look at the complaint linked by the gp (and the people who are lying about here are kind of jaw-dropping).
And I'm not a coddle the kids person. Kids should know all the violent stuff in the world. They should be able to discover it but mere discovery definitely not what's happening in the screenshots I've seen.