But some people are delusional and sufficiently isolated that they will assign affirmation from a machine real meaning in their emotional sensorium.
Which makes me want to build a girlfriend app that is super awesome and supportive and “loving” for months but eventually breaks it off after a chat with another user “leaks” into your context lol. A cheatbot, if you will.
But that’s just sadistic, so I guess I’ll focus on something that has a chance to be actually be a net positive for humanity on more than a Darwinian level instead.
See for example: https://www.reddit.com/r/replika/comments/1305q4v/my_replika...
We'll have to accept the world isn't binary in many situations. There are gradients from "you should know better" to "outright criminal behavior from a device".
Engage this friendly neighbor chatbot about euthanasia, the holocaust, polarized politics and you discover it talks in a friendly way about those subjects. It’s not broken. You did that to yourself.
You can make an innocent typewriter output hate speech if you press its buttons purposefully.