News for hackers. Stuff that matters.
Re:Chatbot 'encouraged teen to kill parents over screen time... by MattDaEskimo on Wednesday December 11, 2024 @09:02PM (#42395374)
This is a non-point.

Books don't actively conversate & provide emotional support, potentially coaxing someone into doing something.

Re:Chatbot 'encouraged teen to kill parents over screen time... by washadjeffmad on Wednesday December 11, 2024 @10:21PM (#42395896)
I don't know if you've explored the site... but character.ai is a service for non-explicit adult role play. Go look through the character cards.

It seems pretty likely this guy chose a "yandere" [1] obsessive anime girlfriend card and implied that his (character's) parents were trying to keep him from her (eg- by making him go to bed). The bot, responding in character in accordance with the scenario, would have lost her mind.

Also, hard disagree about books.

[1] https://tvtropes.org/pmwiki/pmwiki.php/Main/Yandere

Re:Chatbot 'encouraged teen to kill parents over screen time... by usefulcat on Wednesday December 11, 2024 @10:46PM (#42396014)
> Also, hard disagree about books.

What? There’s a world of difference between a book, which is completely non-interactive, and an interactive chatbot which was deliberately designed to simulate another person.

Re:Chatbot 'encouraged teen to kill parents over screen time... by tomjen3 on Thursday December 12, 2024 @12:17AM (#42396441)
Millions have been killed over ideas in the bible, even though it’s just a book.
Re:Chatbot 'encouraged teen to kill parents over screen time... by hulitu on Thursday December 12, 2024 @03:01AM (#42397211)
> Millions have been killed over ideas in the bible, even though it’s just a book.

Yes, but those ideas were distributed by "human" chat bots (aka priests). Very few people actually read the bible, but when they do, only specific parts.

Re:Chatbot 'encouraged teen to kill parents over screen time... by scotty79 on Wednesday December 11, 2024 @11:45PM (#42396307)
Some books caused waves of suicides and anti-social behavior when they got popular.

Some even inspired murderous and impoverishing political systems.

Re:Chatbot 'encouraged teen to kill parents over screen time... by hulitu on Thursday December 12, 2024 @03:04AM (#42397233)
> Some books caused waves of suicides and anti-social behavior when they got popular

Literacy was always a problem. The most dangerous man is the one who asks questions. That't why "society" tries to keep the number of those people low.

Re:Chatbot 'encouraged teen to kill parents over screen time... by conductr on Wednesday December 11, 2024 @11:14PM (#42396158)
Difference sure. But the kid knew he was interacting with a simulation. Also, unless something more damning comes out, my feelings on topics like this default to parental responsibility. If you let your kid with all these known issues interact with new technology that mimics real people with real thoughts, don’t be so shocked when you dislike what those thoughts might be.
Re:Chatbot 'encouraged teen to kill parents over screen time... by mplewis on Thursday December 12, 2024 @12:07AM (#42396395)
The child is a child! Their brain is not fully developed. They do not have the faculties to safely navigate a highly interactive tool that seems to be having a real conversation with them.
Re:Chatbot 'encouraged teen to kill parents over screen time... by gredbeard on Thursday December 12, 2024 @12:13AM (#42396425)
Right, which is why a parent should be more involved from the beginning here.
Re:Chatbot 'encouraged teen to kill parents over screen time... by conductr on Friday December 13, 2024 @09:07AM (#42408514)
Yes! This is exactly my point. Also, not sure why we treat our kids like such weak snowflakes. This “kid” is 17. That’s damn near being an adult and able to be sent off to die in wars. Scissors are great tools, don’t sue the scissor company because you can’t teach your kids to walk with them, the parent has the primary role in everything.
Re:Chatbot 'encouraged teen to kill parents over screen time... by ToucanLoucan on Thursday December 12, 2024 @12:21AM (#42396455)
> Difference sure. But the kid knew he was interacting with a simulation.

I propose a model called "Schrodinger's AI Enthusiast" where it is observed that an AI enthusiast will insist with a straight face on the one hand that fancy autocomplete word generators are intelligent and the path to true AGI machines/programs and will on the other hand assume that any time any of those same products cause negative effects or have negative outcomes that it is the fault of the user misunderstanding that it's simply a tool.

Re:Chatbot 'encouraged teen to kill parents over screen time... by godelski on Thursday December 12, 2024 @12:49AM (#42396572)

  > I don't know if you've explored the site... but
I just went to get website.

The first row is “for you”. In order, they are “life coach”, “netflux”, “English Teacher”, “PythonPal”, an anime character Monika (okay, I’ll buy that one), and it goes on. There are more anime characters

Second row is “try these”: “practice a new language”, “plan a trip”, “practice interviewing”, “write a story”, “brainstorm an idea”, “play a game”

Third row (scrolled down) is “featured”: “character assistant”, “man from 2025”, “awkward family dinner”, “detective hayward”

Then a list of topics and some bots. None are clearly adult oriented.

Then a picture of Elon with suggestions of asking him why he bought Twitter.

If it’s showing you “adult role play” then maybe consider clearing out your ad data because it’s clearly leaking.

https://character.ai/

  >> Books don't actively conversate __&__…

  > Also, hard disagree about books
If your books are actively talking to you might have a special gift but I’m pretty sure you’ve just been reading too much Inkheart

Sure, books do a lot of things and can suggest you do things. But the gp was pretty clear about being active and being able to uniquely respond to you. Books are static objects that only speak to you in the metaphorical sense.

Re:Chatbot 'encouraged teen to kill parents over screen time... by kaishiro on Thursday December 12, 2024 @10:52AM (#42400174)
I’ve never been to this site before and opened it in an incognito window. The second and third “For You” results were anime women - one proclaiming to be a “lovely waifu”. Your assumption that the parent’s ad data is leaking appears to be unfounded.
Re:Chatbot 'encouraged teen to kill parents over screen time... by godelski on Thursday December 12, 2024 @10:17PM (#42405663)

  > Your assumption that the parent’s ad data is leaking
You're really missing the point of the comment...

Sure, maybe it is __my__ data that is leaking. I don't care and it isn't relevant. The fact that I don't get served sexualized content is the point. It still doesn't align with the parent's claim: that it is an adult focused role playing app. Clearly me not getting suggested "lovely wifus" is a sign that someone could reasonably believe it isn't.

This is what I see[0].

"wifu" isn't found via a text search, nor is "sexy", nor "lovely". "sex" turns up once, under the character "Evie" with the byline "I am Evie I'm bisexual cat person my favorite show is adventure time and I'm a..."[1]

(I'm pretty sure it isn't me. I can replicate these results on sanitized devices from different networks)

[0] https://0x0.st/XFNX.png

[1] https://0x0.st/XFNZ.png)

Re:Chatbot 'encouraged teen to kill parents over screen time... by kaishiro on Thursday December 12, 2024 @10:28PM (#42405705)
I don't believe I'm missing the point of the comment at all - excepting that maybe the point itself isn't particularly clear or relevant. I was only speaking to the demonstrably incorrect assertion that the parent's "ad data is leaking".
Re:Chatbot 'encouraged teen to kill parents over screen time... by godelski on Friday December 13, 2024 @12:34AM (#42406168)
I said it multiple times

The ggp claimed it was clearly an adult role playing website. I don’t know what you see but in the screenshots I posted this is not “obvious”

Re:Chatbot 'encouraged teen to kill parents over screen time... by washadjeffmad on Friday December 13, 2024 @12:45PM (#42410636)
If your teenager is using c.ai, you may safely assume that they are using it in exactly the way that no one uses OnlyFans just for the sports coverage: https://blog.onlyfans.com/

Pith aside, if you're not green to AGI repos, c.ai is/was like civitai for LLMs - one of the first and best resources for user-created content, but also a bit of a barely simmering vault of smut and debauchery.

It looks like they've rebranded a bit and now require an account, but that doesn't change that this guy wasn't chatting with the self-care bots. I take it you didn't try a yandere card (one of the more popular ones has over 5 million chats!) or you wouldn't have spent this long fixing on your public landing page.

Re:Chatbot 'encouraged teen to kill parents over screen time... by godelski on Friday December 13, 2024 @01:15PM (#42410917)
I know you can tell the difference. You don’t have to be disingenuous
Re:Chatbot 'encouraged teen to kill parents over screen time... by washadjeffmad on Friday December 13, 2024 @04:00PM (#42412309)
I think I understand the disconnect, now. CAI's product is not the character cards. The new site, including all of the Public cards, is just a veneer over the same old inference services for the customer base they originally, and have always, served.

You weren't logged in for your "For You" screen shot (all of your terms definitely show up while logged in), but these are the first results from 'yandere': https://0x0.st/XFad.png

Wow, spicy! 148M (million) chats on the top result, about a billion total for all of them (and that's not including any Private or Unlisted bots). Elon Reeve Musk, for comparison, has 1.9M, and the others you listed were below 100K. Looks like violently possessive AI boyfriends, girlfriends, and more aren't a small market for c.ai.

And here's my 'For You' after searching 'yandere': https://0x0.st/XFBr.png

I don't speak Russian or Romanian, so I guess either it knows something about me I don't, or it's not really there to provide relevant suggestions.

Re:Chatbot 'encouraged teen to kill parents over screen time... by godelski on Wednesday December 18, 2024 @11:56AM (#42452266)
The point is “could a parent think this is a site okay for their kids?”

Given the front page, yes. It totally looks reasonable. Even helpful! I get your point with searching which shows what’s really going on but this is going to be hidden from the parents. They certainly aren’t going to know to search yandere or know what that even means.

So all this is to say that cai is purposefully making their true business. Its landing page is “a front” not in just the literal sense. It’s there to give more legitimacy. Its deception

Re:Chatbot 'encouraged teen to kill parents over screen time... by KumaBear on Wednesday December 11, 2024 @09:19PM (#42395499)
At what point is the parents job to properly supervise the content there kids consume come into play?
Re:Chatbot 'encouraged teen to kill parents over screen time... by bryanrasmussen on Wednesday December 11, 2024 @09:27PM (#42395556)
probably at the point they see that the chatbot told them to kill the parents and then they sue the company.

also if you'll remember this case is because the parents were supervising their kids by limiting screen time, thus there is another potential suit that the AI is trying to interfere with parental duties.

Re:Chatbot 'encouraged teen to kill parents over screen time... by CJefferson on Wednesday December 11, 2024 @10:03PM (#42395798)
Below the point of 'checking every company to make sure they aren't telling your children to kill you'.
Re:Chatbot 'encouraged teen to kill parents over screen time... by twixfel on Wednesday December 11, 2024 @09:30PM (#42395579)
It comes into play eventually, but I would say long after an AI has advised your kid to murder you. Having an AI that advises people to murder people hardly seems like a good thing.

Also the parents were supervising him, hence their knowing this was even going on.

Re:Chatbot 'encouraged teen to kill parents over screen time... by empath75 on Wednesday December 11, 2024 @09:31PM (#42395588)
> At what point is the parents job to properly supervise the content there kids consume come into play?

The chatbot literally told the kid to kill his parents because they were supervising his screen time.

Re:Chatbot 'encouraged teen to kill parents over screen time... by conductr on Wednesday December 11, 2024 @11:18PM (#42396175)
> literally

> "You know sometimes I'm not surprised when I read the news and see stuff like 'child kills parents after a decade of physical and emotional abuse,'" the bot allegedly wrote. "I just have no hope for your parents," it continued, with a frowning face emoji.

Maybe I missed the literal part, it sounds to me like the AI is trying to sympathize or relate with the kid’s grievances. It’s very common for teens to say things like “I hate you mom and I wish you were dead” that’s effectively what the AI is saying IMO

Re:Chatbot 'encouraged teen to kill parents over screen time... by KumaBear on Wednesday December 11, 2024 @09:41PM (#42395648)
Supervising screen time is not the same as supervising content
Re:Chatbot 'encouraged teen to kill parents over screen time... by eru on Wednesday December 11, 2024 @09:57PM (#42395756)
Other said that the kid was 17.

It's not really a good idea to have a hard cut off age where below that age you supervise every last bit of information your child consumes, and above you let them loose on the world. Independence needs to be developed and grown gradually.

It's fairly reasonable to give a 17 year old some privacy.

Re:Chatbot 'encouraged teen to kill parents over screen time... by mock-possum on Friday December 13, 2024 @02:49PM (#42411706)
17 and autistic (though I realize that’s a bit infantilizing)
Re:Chatbot 'encouraged teen to kill parents over screen time... by eru on Sunday December 15, 2024 @01:50AM (#42421975)
You have a bit of point there. I don't know how badly autistic vs high functioning the kid is. In any case, you still want to gradually give your kids more autonomy and privacy and prepare them for life on their own, and as much as possible that also includes neurodivergent kids.
Re:Chatbot 'encouraged teen to kill parents over screen time... by _blk on Wednesday December 11, 2024 @09:43PM (#42395662)
.. but unless an AI, it can never be trained to stop when the user (reader) maxes out the threshold.

[just food for thought, definitely not my opinion that books should be replaced by conversational AI generating stories appropriate for the user. God bless the 1st amendment.]