What I find disturbing about things like Replika is all your conversations are owned and stored by the company. At least with a real-life partner, there's not a database that someone can query to pull your intimate in-person discussions.
The fact that it's cloud-hosted and company-controlled is the most disturbing part for me.
Can you imagine creating an emotional space for something {subject to Microsoft's TOS and updated unilaterally}?
In addition to the drifting line with intimacy... suddenly you have a privileged position with a huge userbase.
How long until people's agents start recommending {product of the week} in conversations?
Re:Tamagotchi Connection
by RetroTechie on Thursday September 07, 2023 @08:13PM
(#37427530)
Current AI bots tend to be 1 trick dogs. And I have no desire to put emotional investment into a 'brain' that's wired directly to company HQ (especially commercial company). Tool: fine. Buddy: nope. For that I'll stick to the real thing.
That said: as the tech progresses, it wouldn't surprise me if the line between human-human friendships, and human-AI friendships blurs over time.
People are wired for connecting with other beings, but friendships come in many forms. Wouldn't rule out AI-based somethings filling some of those roles.
Business is way ahead of you in intentionally building this -- https://www.vice.com/en/article/93bqbp/can-you-be-in-relatio...
Turns out, mimicking emotional connection is something LLMs are also good at.
Also see here for the moral quandaries users are working through (e.g. do I tell my bot about my real life partner): https://www.reddit.com/r/replika/comments/105g425/replika_as...
The really dystopian part is that Replika regularly pushes updates, which change the sensitive topics boundaries, which has the net effect of rewriting the personality, overnight, of a bot people spent substantial amounts of time getting to know.
We live in crazy times.
(DISCLAIMER for addiction-sensitive readers: maybe don't download an intimate AI chatbot? Your time would probably be better invested with human friends)