Sure, but one of those harms is death and that risk cannot be fully eliminated. Not to mention the ethical concerns of experimenting on animals for the riskiest testing.

Knowing that hundreds of people end their lives /after/ seeking care in our current mental health system, I’m not afraid of exploring other options.

I do believe, however, that developers in this space must not over-represent the capability of their systems and have a duty to strongly warn users that the system can act unpredictably and give harmful advice. Resources to contact health professionals and warnings to contact 911 if there is a life-threatening emergency are also obligatory.

With those disclosures, I don’t see how a self-directed discussion with a language model could be overly harmful.