The privacy aspect is what made me connect local LLMs with therapeutic use. But yeah, AI as it stands today just isn't safe enough. We need nine 9s of safe usage here (99.999999999% safe), or more, for me to actually feel comfortable with the technology.

It would also open up some legal gray areas if it were to happen. Would psychotherapist-patient privilege apply to an LLM box? If the state has a zero day granting them access to a seized "therapy box," it could be more revealing and damaging than anything a human therapist could provide police.