Re:Chatbot Psychosis
by nullc on Tuesday November 25, 2025 @08:36PM
(#46053144)
> no way to prevent the intentional weaponisation of chat/ai
AI psychosis is happening without any intentional weaponization. Sure you can't help intentional bad outcomes but accidental ones are much worse because they don't require a malicious party.
Consider a conman-- a harm to society for sure, but he can only con so many people and he'll only do so in ways that get him money, power, glory, etc. Bad sycophantic AI can con and unlimited number of people and isn't limited to destorying their lives in ways that benefit the AI's owners.
AI psychosis is happening without any intentional weaponization. Sure you can't help intentional bad outcomes but accidental ones are much worse because they don't require a malicious party.
Consider a conman-- a harm to society for sure, but he can only con so many people and he'll only do so in ways that get him money, power, glory, etc. Bad sycophantic AI can con and unlimited number of people and isn't limited to destorying their lives in ways that benefit the AI's owners.