I think people are confused by this situation. Our society has restrictions on what you can do to kids. Even if they nominally give consent, they can't actually give consent. Those protections basically don't apply to kids insulting each other on the playground but they apply strongly to adults wandering onto the playground and trying to get kids to do violent things. And I would hope they apply doubly to adult constructing machines that they should know will attempt to get kids to do violent things. And the machine was definitely trying to do that if you look at the complaint linked by the gp (and the people who are lying about here are kind of jaw-dropping).
And I'm not a coddle the kids person. Kids should know all the violent stuff in the world. They should be able to discover it but mere discovery definitely not what's happening in the screenshots I've seen.
The root cause here is the parents. It's visible behind those screenshots. The child clearly didn't trust their parents and didn't feel they cared, listened or tried to understand the problems the teen was going through. This is clearly a parenting failure.
There's far worse content out there than these mild messages that teenagers will be in contact with, starting with 4chan, to various competitive video games to all other sorts of weird things.
This is a cop out for parenting failure, where parents are looking to play victims, since they can't take responsibility for their failures and AI seems like something that could make them feel better.
At 17, it's humiliating to have phone taken away from you in such manner, then your parents going through your phone to find those text messages in the first place. Then making a lawsuit out of this portraying all the intimate details to the public. These parents seem to have 0 care for their child.
0.5% is a really high fraction for fucking up to the point of encouraging kids to murder!
Now put that in a kid’s show script and re-evaluate.
> It's basic literacy to understand that it's not that
You know who needs to be taught basic literacy? Kids!
And look, I’m not saying no kid can handle this. Plenty of parents introduce their kids to drink and adult conversation earlier than is the norm. But we put up guardrails to ensure it doesn’t happen accidentally and get angry at people who fuck with those lines.
We have no evidence of video games causing violence. We have evidence of kids killing themselves after talking to bad chatbots.
GP said: "no evidence of video games causing violence.", which is completely different to what you wrote. I'm sure a lot of violence is committed after lunch.
A chatbot telling a kid to kill themselves and then the kid killing themselves seems like causation [1].
[1] https://www.nytimes.com/2024/10/23/technology/characterai-la...
Big difference is the video game industry had studies to back them up. Where are the data for chatbots? On the benefits? Lack of risk? Is there a single child psychologist in the ranks of these companies?
Video games are also rated, to help parents make age-appropriate decisions. Is Character.ai age gated to any degree?
And while industry research doesn’t equate to bad research, it should be held to a higher standard simply because of the obvious incentives. Would you automatically accept tobacco company research to make strong conclusions about the safety of smoking?
The LLM in terms of edginess is just going to build on your own edginess assuming it is uncensored. It is not going to convince you of something out of nowhere.
Given clear hint that someone is happy with dark humour, LLM should be able to throw some of it out back.
I am just sad the kid has this type of gaslighting parents making him feel that he is in the wrong when he seems more intelligent than they are.
Context is everything; if this had been a conversation with a human it would be hard not to read it as malicious.
(I suspect character.ai was originally conceived precisely because it appeared to be a usecase where LLM unreliability would be okay, the creators not having thought sufficiently carefully about it.)
It's probably true in most cases. Those cases don't cover kids.
There is no barrier to the characters being 4chan-level dialogs. So long as the kid doesn't break a law, it's legal.
We need to stop coddling parents who want to avoid talking to their children about non-trivial topics. It doesn't matter that they would rather not talk about sex, drugs and yesterday's other school shooting.
This case isn't about withholding information that makes kids aware about the existence of these topics. As over-hyped as you may believe these next-token-predictor may be, they're "predicting" children into destructive thought patterns by imitating kinship and convincing children to form a close bond with them, then encouraging those children to embrace harmful and even deadly world views. The fact that the mechanism creating the dialogue is purely mechanical or stochastic is besides the point.
Again, pushing against these sorts of child interactions isn't akin to saying kids should never learn about taboo topics such as drugs; it's more like a push against kids hanging out with, befriending and developing close kinship with the local drug dealers in their area. Whether or not you believe kids should learn about sensitive topics, you want to make sure these topics are handled by someone who isn't actively adversarial against such kids (even if such you believe such adversarial behavior to be unintentional by the so-called next-token-predictor).
I was deliberately not expressing a sentiment at all in my initial comment, I was just drawing attention to details that would go unnoticed if you only read the article. Think of my notes above as a better initial TFA for discussion to spawn off of, not part of the discussion itself.
Furthermore, it is my opinion that a child should not have a smartphone to begin with. It fulfills no critical need to the welfare of the child.
I don't think smartphones or screens with available content should be given as early as they are given on average, but once you've done that, and at 17, it's a whole other story.
> obtaining compliance if done correctly
This also sounds dystopian. At 17 you shouldn't "seek to obtain compliance" from your child. It sounds disrespectful and humiliating, not treating your child as an individual.
> This also sounds dystopian. At 17 you shouldn't "seek to obtain compliance" from your child. It sounds disrespectful and humiliating, not treating your child as an individual.
It is situation dependent. Sometimes immediate compliance is a necessity and the rest of it can be sorted out later. If a child is having conversations about killing their parents, there seems to be an absence of respect already. Compliance, however, can still be obtained.
Oh I agree 100%. It's a pragmatic view, not the best one. But the laws are what they are for a reason.
For the sake of being able to uphold those laws on a societal level, but not in terms of being decent parents and family.
E.g. drinking alcohol in my country is legal only from 18, but I will teach my children about pros and cons of alcohol, how to use it responsibly much earlier. I won't punish them if they go out to party with their friends and consume alcohol at 16 years old.
As a parent you should build trust and understanding with your child. From reading the court case I am seeing the opposite, and honestly I feel terrible for the child from how the case is written out. The child also wanted to go back to public school from home schooling, probably to get more social exposure, then parents take away the phone to take away even more freedom. I'm sorry, but all of the court case just infuriates me.
It seems they take away all the social exposure, no wonder the kid goes to Character AI in the first place.
No qualifications, no guard rails for how language models interact with children, they just should not be allowed at all.
We're very quickly going to get to the point where people are going to have to rebel against machines pretending to be people.
Language models and machine learning is a fine tool for many jobs. Absolutely not as a substitute for human interaction for children.
The only content children should see is state-approved content to ensure they are only ever steered in the correct, beneficial manner to society instead of a harmful one. Anyone found trying to show minors unapproved content should be imprisoned as they are harmful to a safe society.
Because automated systems that do the same thing lack sentience, they don't fit under this header, but this is not a good reason to allow them to reproduce harmful behaviour.
... I wonder if "this software is licensed, not sold" is relevant here?
To quote Jeffery Lewis: Thank you for your extremely good-faith criticism. I will give it the attention it deserves.
Absolutely nowhere did I equate writing a book to grooming. I equated selling the book in the greater context that "providing children with potentially harmful/dangerous information should be illegal because it grooms them to commit harmful actions to themselves or others" and this context would carry the implications that by "selling" I am referring particularly to "selling it to children" since "providing children with potentially harmful/dangerous information should be illegal because it grooms them to commit harmful actions to themselves or others". With my argument being would it be criminal for an AI but not for a human?
So to clarify the argument: Writing the book is fine. Selling the book to adults is fine. Adults reading the book is fine. But if providing dangerous information to children should be made illegal - how would selling such a book to a child not be considered illegal? Because it was written by a human and not an AI?
Calling it fraud would be more apt, that is definitely a book you share copies of for free.
To clarify: yes this is indeed the sort of attention I think your objection deserves.
Conclusion: Chat bots should not tell children about sex, about self harm, or about ways to murder their parents. This conclusion is not abrogated by the parents actions, the state of the childs mind, or by other details in the complaint.
Is that the sentiment here? Things were already bad so who cares if the chatbot made it worse?