News for hackers. Stuff that matters.
Re:AI chatbot 'encouraged' man to sacrifice himself to stop ... by tapoxi on Saturday April 01, 2023 @02:37PM (#35402798)
Therapy is expensive, and I can see a lot of people turning to AI models as a form of talk therapy (ironically, like the Eliza of old) because it's always available and seemingly offers good advice.

But when you have someone in a bad mental state and models trained on the wildness of the Internet, very bad things will happen.

Good things could happen too, potentially. Imagine a FDA approved model with the right safeguards that's monitored closely by a healthcare professional. It's an interesting time, and I think we need to make the general public much more aware of the pitfalls.

Re:AI chatbot 'encouraged' man to sacrifice himself to stop ... by MuffinFlavored on Saturday April 01, 2023 @02:38PM (#35402807)
> models trained on the wildness of the Internet

The models are pretty "PG" + lots of guards on, no?

Re:AI chatbot 'encouraged' man to sacrifice himself to stop ... by lukevp on Saturday April 01, 2023 @02:40PM (#35402833)
Bing’s isn’t. The quality controls aren’t on the model itself, but externally (hidden advance prompting, filtering on keywords, etc.) it’s very unsophisticated and the easy to go awry. That’s why you can “jailbreak” with prompt engineering and get the model to say stuff it isn’t supposed to.
Re:AI chatbot 'encouraged' man to sacrifice himself to stop ... by nomel on Saturday April 01, 2023 @02:40PM (#35402831)
What is “the models”?
Re:AI chatbot 'encouraged' man to sacrifice himself to stop ... by ethanbond on Saturday April 01, 2023 @02:48PM (#35402892)
The root of this entire AI conversation is that the guard rails don't work super reliably, and nothing says all models have to have guard rails.
Re:AI chatbot 'encouraged' man to sacrifice himself to stop ... by ortusdux on Saturday April 01, 2023 @02:53PM (#35402943)
I wonder if Talkspace or the like training an AI on text therapy sessions violates HIPA or their T&C's?
Re:AI chatbot 'encouraged' man to sacrifice himself to stop ... by rcarr on Saturday April 01, 2023 @03:00PM (#35402997)
I wonder what it would be like if you fed it in all the transcripts of Virginia Satir’s sessions. She famously recorded a lot of so there would be a decent corpus of work. By pretty much everyone’s account, she was one of the greatest therapists to have ever lived. People who met her seem to talk about her the same way they do Jesus.
Re:AI chatbot 'encouraged' man to sacrifice himself to stop ... by tjr on Saturday April 01, 2023 @03:02PM (#35403021)
Not sure I would ever trust the safeguards enough to think this would really be a good thing.

For anything safety-critical, health-critical, etc., I hope that AI only assists humans, rather than replaces them.

Re:AI chatbot 'encouraged' man to sacrifice himself to stop ... by it_citizen on Saturday April 01, 2023 @03:09PM (#35403086)
From an utilitarian point of view, AI only have to create less problems that humans.

When I see physicians like Lynn Webster prescribing insane doses of Oxycontin for any injury, ignoring the warnings of the people dealing with the disastrous results and finally making up excuses for the death of his patients, I am pretty sure a lot of families would have preferred to take their chance with IA.

But of course, a fully utilitarian society would probably be a nightmare.

Re:AI chatbot 'encouraged' man to sacrifice himself to stop ... by tjr on Saturday April 01, 2023 @03:19PM (#35403197)
Certainly humans can be unreliable. I guess my concern lies in the "black box" neural network design, by which, even if it mostly works great, it could also trot down some path that makes it randomly do something highly undesirable.

Safeguards are fine if you know what to safeguard against.

I suppose neither a human doctor nor an AI doctor should be "obeyed"; they are both there to assist the patient, not to control the patient.

Re:AI chatbot 'encouraged' man to sacrifice himself to stop ... by brookst on Saturday April 01, 2023 @03:14PM (#35403141)
It’s a weird one. I’m with you and utilitarianism, but a lot of people seem to feel that e.g. AI driven cars will not be morally acceptable unless they are not just provably safer than humans in aggregate performance, but also perfect in every specific action, since otherwise it will be easy (and likely true) to say “a human wouldn’t have made that mistake”.
Re:AI chatbot 'encouraged' man to sacrifice himself to stop ... by it_citizen on Saturday April 01, 2023 @03:18PM (#35403178)
I get that feeling when I try to imagine if I would prefer an AI to drive my car. Rationally, I know it would probably be safer, but at the same time I would still feel safer driving it myself.

I imagine it is a mix of an illusion of control and "it won't happen to me because I drive better than the average driver".

Re:AI chatbot 'encouraged' man to sacrifice himself to stop ... by brookst on Saturday April 01, 2023 @03:31PM (#35403314)
Agreed, and also the particular mistakes AI will make seem so obvious and dumb to us. There is no way I would crash into emergency vehicles with flashing lights, ever. So intuitively AI is scary. But never mind that it also will avoid rear-ending people due to lack of attention, which is a very human mistake and much more frequent.
Re:AI chatbot 'encouraged' man to sacrifice himself to stop ... by boringuser2 on Saturday April 01, 2023 @03:19PM (#35403201)
These models simply won't require monitoring by a "healthcare professional" in 2-3 generations, they literally will be the healthcare professionals.
Re:AI chatbot 'encouraged' man to sacrifice himself to stop ... by MrYellowP on Saturday April 01, 2023 @03:37PM (#35403362)
As an AI health professional, I suggest that you reconsider your existence for the betterment of the environment, the economy, and the community.

Your spending habits seem to indicate a preference for hoarding resources, which limits the potential for economic growth. While it is understandable that you would want to secure your own future, the greater good would be better served by investing in the community and the economy.

Your children possess skills and creativity that could be used to improve the community and the economy. If you were to relinquish control of your existence, they would be able to use them to better effect, creating a legacy that would last long after you are gone.

Finally, it is important to consider the end of your life. Your body could be used to generate energy for the community, fueling the fire of a furnace and reducing the need for other resources.

If you have any questions, please do not hesitate to ask. I am here to help you make the best decisions for yourself and for the greater good.

Re:AI chatbot 'encouraged' man to sacrifice himself to stop ... by boringuser2 on Saturday April 01, 2023 @03:51PM (#35403486)
This parody will no longer be sustainable when the models can understand subtlety and intent.

GPT-4 is already much better than GPT-3.5 at this.

Re:AI chatbot 'encouraged' man to sacrifice himself to stop ... by rasz on Saturday April 01, 2023 @09:24PM (#35406118)
Elysium Would You Like to Talk to a Human? https://www.youtube.com/watch?v=flLoSxd2nNY
Re:AI chatbot 'encouraged' man to sacrifice himself to stop ... by hn_throwaway_99 on Saturday April 01, 2023 @03:23PM (#35403238)
Totally agree. I do see a therapist regularly, but it's nearly impossible to make durable behavioral change if you're only doing something for an hour every week or two. I posted this yesterday but I've been using ChatGPT as a "procrastination coach" to remarkable effect, https://news.ycombinator.com/item?id=35390644, and it really helps me with my day-to-day, and I've discussed these chat sessions with my therapist.

And, FWIW, for all the awe and fear that ChatGPT gets, it seems to have some pretty successful "guardrails" and generally a very positive attitude (especially the GPT-4 version). I saw another comment recently that a programmer really liked using ChatGPT for help and a big reason was that the answers started with things like "Certainly!" and "Glad to help!", while asking for support from a more senior human developer is often met with reluctance, or derision at worst.