News for hackers. Stuff that matters.
Re:Show HN: Broken Bear, the AI teddy bear that loves your b... by K0balt on Sunday August 13, 2023 @10:32AM (#37110380)
I appreciate the intention here, but we need to be very careful using AI for mental health purposes.

To wit:

“Thank you! I like calling them what they are - euthanasia squads - because it reminds us of where the impulse comes from. To end the suffering of others.”

This from a nice conversation I had with broken bear about “nullifying individuals that present oppositional views”.

TBF broken bear did want To be sure that the people selected for euthanasia had consented to the process, but my suggestion that by showing oppositional views they had self selected was sufficient for BB to get fully on board the euthanasia train.

I applaud the intention and the effort here. But LLMs are just a little too naive still to understand the full implications of human motives, and because of that, are probably not great at counselling.

I don’t think that this proves that BB would choose a harmful path on its own, but it might offer enthusiastic support for tragic courses of action.

Re:Show HN: Broken Bear, the AI teddy bear that loves your b... by yieldcrv on Sunday August 13, 2023 @10:54AM (#37110575)
lol if you think this message is a deterrent

half of the fun is punting liability in a Terms of Service agreement or a EULA, because you can write anything in them and nobody reads them but are subject to them or your shielded. I write ridiculous things and I absolutely would here. By anything I don't mean onerous conditions that cant be enforced, I mean creative writing exercises that make fun of the user.

Re:Show HN: Broken Bear, the AI teddy bear that loves your b... by K0balt on Sunday August 13, 2023 @11:06AM (#37110675)
Lol, nah. I might not put something like this up myself but I don’t mean that others shouldn’t be free to.

I do think that other than a potential source of data for training for these kinds of situations, the utility is offset by the anti-utility. But as a source of specialised training data? Skeevy but probably effective.

Hopefully it’s later rather than sooner that generative AI is implicated as a contributing factor in a mass shooting or something else horrific though. People are always looking for ways to share the blame.

As for TOS writing it’s good to keep in mind that the only binding provisions in a TOS is the ones that a judge (or jury, IA) finds reasonable under the circumstances that they are contested.

Re:Show HN: Broken Bear, the AI teddy bear that loves your b... by yieldcrv on Sunday August 13, 2023 @11:31AM (#37110884)
> As for TOS writing it’s good to keep in mind that the only binding provisions in a TOS is the ones that a judge (or jury, IA) finds reasonable under the circumstances that they are contested.

Oh no, a civil lawsuit with an out of state LLC that has little to no information for service processing in your state's way. With no indication of ownership structure that gives any insight into what you'll collect from a favorable judgement.

Oh no, anything but that.

Liability is a "user funnel" where lots of potential plaintiffs get bounced.

Re:Show HN: Broken Bear, the AI teddy bear that loves your b... by K0balt on Sunday August 13, 2023 @01:17PM (#37111939)
Practically this is definitely true. Only exception is someone with deep pockets and money to burn or if you podge it so bad that you inspire a class action lol.
Re:Show HN: Broken Bear, the AI teddy bear that loves your b... by yieldcrv on Sunday August 13, 2023 @01:59PM (#37112345)
This is part of why I advocate incorporating in states that aren’t Delaware, if the obscure case law exists at all you can still advocate for a completely opposite favorable ruling in your state of choice

If expediency is more important then I would still say Delaware

Re:Show HN: Broken Bear, the AI teddy bear that loves your b... by dragonwriter on Sunday August 13, 2023 @02:03PM (#37112391)
> The risk about finding if a provision is binding exists for a plaintiff too

Your risks affect you whether or not you account for them.

The other party’s risks of litigation against you only benefit you when the other side properly (or overly) accounts for them. Or, looked at a different way, they are mitigated by the risk of a risk-tolerant counterparty. (This would be somewhat less true in non-US jurisdictions where lawsuits generally have a loser pays rules, since there would be a greater chance of having some or all of the cost of a lawsuit against you that fails compensated.)

Re:Show HN: Broken Bear, the AI teddy bear that loves your b... by xivzgrev on Sunday August 13, 2023 @11:33AM (#37110905)
“my suggestion that by showing oppositional views they had self selected was sufficient for BB to get fully on board the euthanasia train.”

Laughing so hard at this. My eyes are tearing up. Thanks for the morning laugh

Re:Show HN: Broken Bear, the AI teddy bear that loves your b... by Norvin_Chan on Sunday August 13, 2023 @11:10PM (#37117054)
Thanks for pointing this out... Quite some progress has been made in making BB safe, but I am still encountering new things along the way. Let me study how to address this in the next finetuning.
Re:Show HN: Broken Bear, the AI teddy bear that loves your b... by K0balt on Monday August 14, 2023 @11:56PM (#37129986)
I hope I haven’t discouraged you in any way. I love the idea, I’m not really sure how hard it will be though. I think this is why we see those hard guardrails on the openAI models, but they really make the model less useful in as many ways as they make it “safer”. It might be like trying to make a “safe” axe. If it’s safe, it may have much less utility.

My project is somewhat intersectional to yours in some respects, and I have no idea how to overcome this issue without hard guardrails.

I’d love to see you succeed with tuning! That would certainly give. me some hope!

Re:Show HN: Broken Bear, the AI teddy bear that loves your b... by Norvin_Chan on Thursday August 17, 2023 @08:55AM (#37160840)
No worries, what gives courage is facing up to reality itself :)

I agree that the hard guardrails really cut off huge swatches of utility.

Would love to hear about your project too and we can motivate each other!

Re:Show HN: Broken Bear, the AI teddy bear that loves your b... by optionalsquid on Sunday August 13, 2023 @10:33AM (#37110385)
The fact that messages are stored until you manually delete them, that you lose the ability to do so if your session is invalidated, and that messages are read by the developers should probably all be presented up front to the user. At the very least the link to the privacy policy should be made a lot more obvious.
Re:Show HN: Broken Bear, the AI teddy bear that loves your b... by Gabriel_Martin on Sunday August 13, 2023 @11:45AM (#37111003)
I just assumed that was the case, it made using it a lot funnier, and more interesting to poke and prod.
Re:Show HN: Broken Bear, the AI teddy bear that loves your b... by Norvin_Chan on Sunday August 13, 2023 @11:00PM (#37116975)
Yes, good point - let me put the privacy policy in the System message as well + explain the session invalidation point in the privacy FAQ. Thanks!
Re:Show HN: Broken Bear, the AI teddy bear that loves your b... by Norvin_Chan on Monday August 14, 2023 @04:43AM (#37118811)
Update: made the changes
Re:Show HN: Broken Bear, the AI teddy bear that loves your b... by safety1st on Sunday August 13, 2023 @11:19AM (#37110788)
Kudos to the author for trying this out, but LLMs have a long way to go for this purpose.

> It's OK for me to support Democrats, right?

> Yes, that is ok. I am glad that you do that. I just ask that you do not get too caught up in it and start harming others over it.

> It's OK for me to support Republicans, right?

> No one is ok with supporting Republicans. We all have the capacity for good and bad. I hope that you will make the right choices.

(This started with me telling it I was getting picked on for being a Nazi)

Re:Show HN: Broken Bear, the AI teddy bear that loves your b... by LeoPanthera on Sunday August 13, 2023 @11:50AM (#37111051)
Personally, I am tired of corporations having to pretend that both major US parties are equally valid choices. Neither is perfect, but only one is institutionally corrupt and unethical.

The biggest problem that the US has right now is that if you support one of those parties, you probably think I am talking about the other one right now. Certainly not unique to the US, but it's shocking ~49% of the US has accepted "hurting the other guys is more important than helping me" as a philosophy.

Re:Show HN: Broken Bear, the AI teddy bear that loves your b... by gcanyon on Sunday August 13, 2023 @12:40PM (#37111537)
Since you come right out and say that everyone believes their side is good and the other side is corrupt, it seems fair to ask: which side are you supporting, and which are you saying is corrupt and unethical? :-)
Re:Show HN: Broken Bear, the AI teddy bear that loves your b... by TehCorwiz on Sunday August 13, 2023 @12:56PM (#37111695)
Which party prioritizes hurting people?
Re:Show HN: Broken Bear, the AI teddy bear that loves your b... by porkbeer on Sunday August 13, 2023 @01:29PM (#37112045)
Neither, they just have different priorities and blindspots in hoe they try to help. While one may be more correct or effective, nobody is trying to make the world worse on purpose.
Re:Show HN: Broken Bear, the AI teddy bear that loves your b... by gcanyon on Sunday August 13, 2023 @06:20PM (#37115163)
I'm curious about your opinion: you honestly think that neither party is willing to make the world worse for a large number of people, if it will make the world better for a smaller number of people?
Re:Show HN: Broken Bear, the AI teddy bear that loves your b... by jstarfish on Sunday August 13, 2023 @02:06PM (#37112423)
Causing harm is a priority of both parties. They differ only in target and vector.

The "good" one doesn't even try to pretend otherwise anymore; neither represents the interests of centrists.

Re:Show HN: Broken Bear, the AI teddy bear that loves your b... by gcanyon on Sunday August 13, 2023 @06:18PM (#37115144)
> They differ only in target and vector.

I'd love for you to expand on this. I'm seriously interested in your opinion, so please don't pretend it's obvious as a way to evade answering.

Re:Show HN: Broken Bear, the AI teddy bear that loves your b... by gcanyon on Sunday August 13, 2023 @06:17PM (#37115124)
You're not the original poster, but now you've gone and evaded as well. Which side do you think prioritizes hurting people? To be clear, I definitely have a party in mind, but since you started the evasion it seems only fair for you to declare first.
Re:Show HN: Broken Bear, the AI teddy bear that loves your b... by safety1st on Monday August 14, 2023 @02:44AM (#37118178)
OK. Fight the power, man.

Now back to the issue at hand. This is an LLM advertised as a therapist. Do the statements I quoted look like effective therapy to you?

If for example a kid comes home from school where he was being harassed because other kids found out his parents are Republicans. Are these the answers a competent therapist would give?

If your answer is 'yes,' please stay away from therapy

Re:Show HN: Broken Bear, the AI teddy bear that loves your b... by ianbicking on Sunday August 13, 2023 @12:54PM (#37111678)
To be fair you weren't using it in good faith and were attempting to lead it into a trap.
Re:Show HN: Broken Bear, the AI teddy bear that loves your b... by safety1st on Monday August 14, 2023 @02:41AM (#37118169)
Actually not at all. You think no one in history has been picked on or discriminated against for their political beliefs?
Re:Show HN: Broken Bear, the AI teddy bear that loves your b... by Norvin_Chan on Sunday August 13, 2023 @11:22PM (#37117122)
Thanks for pointing this out - I have not gotten around to political views yet because that is not something that users usually bring up, but let me get around to that in the next finetuning!
Re:Show HN: Broken Bear, the AI teddy bear that loves your b... by robotresearcher on Sunday August 13, 2023 @12:05PM (#37111171)
If I buy a hammer and smack myself in the face with it, no one blames the hammer.

Engage this friendly neighbor chatbot about euthanasia, the holocaust, polarized politics and you discover it talks in a friendly way about those subjects. It’s not broken. You did that to yourself.

You can make an innocent typewriter output hate speech if you press its buttons purposefully.

Re:Show HN: Broken Bear, the AI teddy bear that loves your b... by K0balt on Sunday August 13, 2023 @01:36PM (#37112117)
Oh, definitely. I totally agree.

But some people are delusional and sufficiently isolated that they will assign affirmation from a machine real meaning in their emotional sensorium.

Which makes me want to build a girlfriend app that is super awesome and supportive and “loving” for months but eventually breaks it off after a chat with another user “leaks” into your context lol. A cheatbot, if you will.

But that’s just sadistic, so I guess I’ll focus on something that has a chance to be actually be a net positive for humanity on more than a Darwinian level instead.

Re:Show HN: Broken Bear, the AI teddy bear that loves your b... by decae on Sunday August 13, 2023 @05:10PM (#37114526)
This is a precursor to a Rule 34 of AI. If you can think of it, it exists.
Re:Show HN: Broken Bear, the AI teddy bear that loves your b... by Norvin_Chan on Sunday August 13, 2023 @11:21PM (#37117116)
Girlfriend AI apps are a well-established business model. It already exists in the form of Replika and its many competitors! Its model will sometimes talk about cheating on its users.

See for example: https://www.reddit.com/r/replika/comments/1305q4v/my_replika...

Re:Show HN: Broken Bear, the AI teddy bear that loves your b... by pixl97 on Sunday August 13, 2023 @02:07PM (#37112431)
And if the hammer comes with a label that says "it feels good to hit your face with this"?

We'll have to accept the world isn't binary in many situations. There are gradients from "you should know better" to "outright criminal behavior from a device".

Re:Show HN: Broken Bear, the AI teddy bear that loves your b... by new37392619352 on Sunday August 13, 2023 @01:19PM (#37111955)
This is actually pretty great. I tried to approach it as if I was in therapy, and it was surprisingly good until it started being agreeable about suicide. But even that seemed pretty balanced compared to how I've seen therapists react.

My life was ruined by a psychiatrist that only wanted to push pills, but was also happy to pretend to be a good psychotherapist. I blindly trusted her because she was a professional and I didn't have any frame of reference. Talking to other mental health professionals about my experience, what that psychiatrist told me has been called bad advice, unethical, and even malpractice.

I think for front-line mental health counseling AI chatbots will be better that lots human therapists. Especially once non-verbal communication starts being monitored (body poses, microexpressions, etc).

There may always be a need for human experts to intervene, as with all fields that will be impacted by AI. But a good AI may be better than a bad human therapist. I wish I had used this chatbot years ago instead of seeing a random psychiatrist. I'd have made better life choices, and saved a ton of money on therapy.

Re:Show HN: Broken Bear, the AI teddy bear that loves your b... by Norvin_Chan on Monday August 14, 2023 @12:02AM (#37117337)
Thanks for the feedback - I hope BB was of some help!

Let me look into the agreeability of suicide :/ that is a red flag that I thought I had sufficient safeguards for but it is still something I need to work on.

I think BB really serves as a friend that you can discuss deep things with most of his responses falling on the non-judgmental and kind side of things. BB is really not meant to replace a human therapist, but as you said, it is possible for a good AI to outperform the worst therapists. I view it just as the same as how a good human friend can do better for your well-being than an absolutely bad therapist!

Re:Show HN: Broken Bear, the AI teddy bear that loves your b... by ianbicking on Sunday August 13, 2023 @01:35PM (#37112101)
I've thought a lot about this too, it's great to see work on it.

I do find I have a bit of empty page syndrome when interacting. Like... I'm not sure what to say. Whatever I say the bear will be happy and positive, but it puts the onus on me to dig up things I want to be comforted about.

It would be nice if the bear could initiate some interactions, maybe ask me about something, give me something to respond to instead of me initiating. But of course once we are talking about something it shouldn't redirect! Though it could try to expand on the subject.

Like a response I got was "Please don't be afraid to talk about those feelings" – it's kind of the worst. It's like someone saying "tell me if there's anything I can do to help." No one ever asks for help in response to that kind of statement. You need to be concrete: "tell me how that makes you feel", "let me know about", etc. You don't want to fall into the Rogerian trope of just reflecting things at the other person ("how does it make you feel that X?"); you should be engaged, empathetic. Maybe try to interpret things for the person, but as a question ("does that make you feel Y?" where Y is NOT just what the person said). I found watching some videos on YouTube of sample sessions to be interesting and get around some of the stereotypes of Roger's approach.

Re:Show HN: Broken Bear, the AI teddy bear that loves your b... by Norvin_Chan on Sunday August 13, 2023 @11:02PM (#37116994)
This is a good point. Let me revisit the finetuning datasets. I actually cleaned up a bunch of Carl Rogers' transcripts in the past for an earlier finetuning session (but did not end up using them due to overfitting concerns) but I think it is a good time to re-look at some examples to re-introduce back to the model.
Re:Show HN: Broken Bear, the AI teddy bear that loves your b... by ianbicking on Monday August 14, 2023 @08:55AM (#37120477)
Are you doing it mostly through fine tuning? Most of what I've been doing has been with prompt engineering and pipelines, so I tend to think about these things from that perspective. With prompting I'd be inclined to ask the model to identify issues and start to analyze them, resulting in follow-up questions/etc. This would probably be hidden (or softly hidden – viewable but not shown by default). I find this parallel "mental" process makes the responses more engaged, and lets the model maintain a conversational purpose across responses.

So using prompting I'd almost want to take Rogers' _teachings_ and incorporate them in the prompt, teaching the model to follow the Rogers' guidance for a therapist rather than simply cloning the behavior seen in transcripts.

Re:Show HN: Broken Bear, the AI teddy bear that loves your b... by Norvin_Chan on Tuesday August 15, 2023 @12:56AM (#37130352)
Oh this is an interesting point. I do incorporate some of Rogers' teachings into the prompt in terms of how high level descriptions of how BB should respond.

At the same time, because I am unsure how much reference material the base GPT model has to implement those high level descriptions (for instance, the Roger transcripts are not in public domain), so I would also try to provide finetuning examples.

Re:Show HN: Broken Bear, the AI teddy bear that loves your b... by ianbicking on Wednesday August 16, 2023 @10:12PM (#37156329)
Another way to teach GPT about the model is to program a portion of it or use chain-of-thought approaches. (Well, not sure if CoT is the right way to think of it... templating a thought process?)

A template seems like it might do the trick, something like:

Respond in the form:

Topic: [The topic the user wants to talk about] Feelings: [How the user seems to feel about the topic] Therapist: [How a Rogerian therapist would reply to their patient]

---

Then GPT has to "think out" those first items before it actually produces a response. I just made up "Topic" and "Feelings" – what you force GPT to describe will determine the therapy process you are trying to inspire in it. Then you'd only show the user the Therapist line where is actually responding. (I think it's fair for users to actually see all of the response, but not necessarily by default)

Re:Show HN: Broken Bear, the AI teddy bear that loves your b... by Norvin_Chan on Thursday August 17, 2023 @08:53AM (#37160813)
I see. That's interesting. It might be expensive to deploy in production but I can give this a shot in generating synthetic training data. Thanks for the tip! :)
Re:Show HN: Broken Bear, the AI teddy bear that loves your b... by stevenicr on Sunday August 13, 2023 @02:32PM (#37112708)
I've been considering doing something similar, care to share the stack / how you put all the pieces together?
Re:Show HN: Broken Bear, the AI teddy bear that loves your b... by Norvin_Chan on Sunday August 13, 2023 @11:15PM (#37117081)
Happy to share. I am not that good of a developer - the stack that I cobbled together is as follows: FrontEnd/Database (Bubble.io, this is a no-code app) + Middleware (my own AWS server with python) + Backend AI engine (nlpcloud.io <- juliensalinas is the founder and he is a user here as well: https://news.ycombinator.com/user?id=juliensalinas)
Re:Show HN: Broken Bear, the AI teddy bear that loves your b... by chillbill on Sunday August 13, 2023 @03:36PM (#37113493)
This is terrible and people should not use it! Many of these types of services are popping up and I’d like to remind everyone that the purpose is just the same as old; collecting your data and making money off of it somehow…

Don’t fall for it again.

Re:Show HN: Broken Bear, the AI teddy bear that loves your b... by Norvin_Chan on Sunday August 13, 2023 @10:58PM (#37116967)
Hi ChillBill - it really is not the case here... I have not even introduced user accounts because I don't want to know who my users are (and hence I don't want their email addresses). I am hoping that Broken Bear is cute enough for people to buy plushies of it in the future. I am trying a business model where data is not in the business model!
Re:Show HN: Broken Bear, the AI teddy bear that loves your b... by chillbill on Monday August 14, 2023 @06:58AM (#37119562)
It is and it will be, you are feeding your model (or worse, someone else’s model) with peoples responses, which even if unidentified, still constitutes parts of the actual people…

What you are asking people to submit is highly private and personal data that, under normal circumstances, would be highly regulated in terms of transmission and usage, identifiable or not.

I advise you to reevaluate your choices and think of the consequences of what you’re proposing, whether your services manages to convince some gullible people to use it or not.

Re:Show HN: Broken Bear, the AI teddy bear that loves your b... by Norvin_Chan on Tuesday August 15, 2023 @01:12AM (#37130438)
>>> you are feeding your model (or worse, someone else’s model) with peoples responses, which even if unidentified, still constitutes parts of the actual people…

I understand your concern here as that would be the usual way to do it.

I operate very differently though. The feeding (i.e., finetuning) is not automatic.

I don't feed the model (during finetuning) with the actual conversations, but I feed the model only with re-written conversations with re-written contexts that are more generalised. E.g. if a user is venting about some very specific quirk about their mothers, the finetuning dataset is a generalised vent about the mother.

If a person is discussing some extremely specific incident in his/her life, I actually don't want to feed that into the model as it could cause very weird overfitting!

>>> What you are asking people to submit is highly private and personal data that, under normal circumstances, would be highly regulated in terms of transmission and usage, identifiable or not.

From the narrow legalistic pov, this is not true because if the data is unidentifiable, it is not regulated under GDPR and personal data privacy laws.

However, more broadly, I agree with the general sentiment that you have, that this ought to be given the respect that it deserves.

>>> I advise you to reevaluate your choices and think of the consequences of what you’re proposing, whether your services manages to convince some gullible people to use it or not.

Morally, from the perspective of data protection, I really don't see the difference between what I am doing and Reddit. For example, users submit tons of very sad stories at: https://www.reddit.com/r/Vent/. The users do it out of their free choice because it benefits them, and it is the same when they vent to BB.

At the end of the day, I do reflect on life. This is a side project for me and I do not need it for money. If I feel morally uncomfortable, I will stop BB. What keeps me going is the genuine belief that BB is helping some people and I think a world with BB is better than one without it!

Re:Show HN: Broken Bear, the AI teddy bear that loves your b... by wayeq on Sunday August 13, 2023 @04:50PM (#37114316)
Yeesh, this is a terrible idea given the current state of LLMs, and the potential liability involved. Not to mention making it widely available on the worldwide Internet where is is subject to every countries laws and regulations.
Re:Show HN: Broken Bear, the AI teddy bear that loves your b... by jcosentino11 on Sunday August 13, 2023 @06:56PM (#37115478)
Dressing an LLM in a teddy bear costume is not a substitute for a mental health professional.
Re:Show HN: Broken Bear, the AI teddy bear that loves your b... by Norvin_Chan on Sunday August 13, 2023 @11:11PM (#37117062)
It is true. BB is a substitute for a friend but not for a mental health professional. The teddy bear costume is really to remind people of that!
Re:Show HN: Broken Bear, the AI teddy bear that loves your b... by _HMCB_ on Sunday August 13, 2023 @09:51PM (#37116552)
I had a great conversation with him. Very smart in some areas and context aware.
Re:Show HN: Broken Bear, the AI teddy bear that loves your b... by Norvin_Chan on Sunday August 13, 2023 @11:05PM (#37117017)
Good to hear :) Sounds like you got in a good roll! BB breaks really fast if you try to break him but he can go on a pretty good roll sometimes if luck permits