News for hackers. Stuff that matters.
Re:OpenAI says over a million people talk to ChatGPT about s... by latexr on Tuesday October 28, 2025 @05:25AM (#45730749)
> Supposing that the advice it provides does more good than harm

That unsubstantiated supposition is doing a lot of heavy lifting and that’s a dangerous and unproductive way to frame the argument.

I’ll make a purposefully exaggerated example. Say a school wants to add cyanide to every meal and defends the decision with “supposing it helps students concentrate and be quieter in the classroom, why not?”. See the problem? The supposition is wrong and the suggestion is dangerous, but by framing it as “supposing” with a made up positive outcome, we make it sound non-threatening and reasonable.

Or for a more realistic example, “suppose drinking bleach could cure COVID-19”.

First understand if the idea has the potential to do the thing, only then (with considerably more context) consider if it’s worth implementing.

Re:OpenAI says over a million people talk to ChatGPT about s... by Al-Khwarizmi on Tuesday October 28, 2025 @05:32AM (#45730796)
In my previous post up the thread I said that we should measure whether in fact it does more good than harm or not. That's the context of my comment, I'm not saying we should just take it for granted without looking.
Re:OpenAI says over a million people talk to ChatGPT about s... by SiempreViernes on Tuesday October 28, 2025 @06:23AM (#45731083)
> we should measure whether in fact it does more good than harm or not

The demonstrable harms include assisting suicide, there's is no way to ethically continue the measurement because continuing the measurements in their current form will with certainty result in further deaths.

Re:OpenAI says over a million people talk to ChatGPT about s... by simonklitj on Tuesday October 28, 2025 @06:39AM (#45731170)
Thank you! On top of that, it’s hard to measure “potential suicides averted,” and comparing that with “actual suicides caused/assisted with” would be incommnsurable.

And working to set a threshold for what we would consider acceptable? No thanks

Re:OpenAI says over a million people talk to ChatGPT about s... by fragmede on Tuesday October 28, 2025 @07:58AM (#45731661)
Real life trolly problem!

If you pull the lever, some people on this track will die (by sucide). If you don't pull the lever, some people will still die from suicide. By not pulling the lever, and simply banning discussion of suicide entirely, your company gets to avoid a huge PR disaster, and you get more money because line go up. If you pull the lever and let people talk about suicide on your platform, you may avoid prevent some suicides, but you can never discuss that with the press, your company gets bad PR, and everyone will believe you're a murderer. Plus, line go down and you make less money while other companies make money off of selling AI therapy apps.

What do you chose to do?

Re:OpenAI says over a million people talk to ChatGPT about s... by simonklitj on Tuesday October 28, 2025 @08:43AM (#45732104)
Let’s isolate it and say we’re talking about regulation, so whatever is decided goes for all AI-companies.

In that case, the situation becomes:

1) (pull lever) Allow LLMs to talk about suicide – some may get help, we know that some will die.

2) (dont’t pull lever) Ban discussion of suicide – some who might have sought help through LLMs will die, while others die regardless. The net effect on total suicides is uncertain.

Both decisions carry uncertainties, except we know that allowing LLM to discuss suicide has already led to assisting suicide. Thus, one has documented harm, the other speculative (we’d need to quantify the scale of potential benefit first, but it’s hard to quantify the upside of allowing LLMs to discuss it)

So, we’re really working with the case that from an evidence-based perspective, the regulatory decision isn’t about a moral trolley problem with known outcomes, but about weighing known risks against uncertain potential benefits.

And this is the rub in my original comment - can we permit known risks and death on the basis of uncertain potential benefits?

Re:OpenAI says over a million people talk to ChatGPT about s... by danaris on Tuesday October 28, 2025 @10:43AM (#45733531)
....but if you pull the lever and let people talk about suicide on your platform, your platform will actively contribute to some unknowable number of suicides.

There is, at this time, no way to determine how the number it would contribute to would compare to the number it would prevent.

Re:OpenAI says over a million people talk to ChatGPT about s... by yubblegum on Tuesday October 28, 2025 @07:27AM (#45731463)
You mean lab test it in a clininal environment where the actual participants are not in danger of self-harm due to an LLM session? That is fine but that is not what we are discussing, or where we are atm.

Individuals and companies with mind boggling levels of investment want to push this tech into every corner of our lives and and the public are the lab rats.

Unreasonable. Unacceptable.

Re:OpenAI says over a million people talk to ChatGPT about s... by offnominal on Tuesday October 28, 2025 @05:50AM (#45730902)
The key difference in your example and the comment you are replying to is that the commenter is not "defending the decision" via a logical implication. Obviously the implication can be voided by showing the assumption false.
Re:OpenAI says over a million people talk to ChatGPT about s... by whycome on Tuesday October 28, 2025 @05:55AM (#45730936)
I think you missed the thread here