People have been developing fake relationships and playing social games which end in harm and heartbreak for millenia but now that we're talking to a computer it's a problem.

Almost like people drank coca tea for millennia without problems, but crack cocaine clearly does have problems.

Lots of drugs have been problematic before more modern refinement methods, my problem was the framing of chatbots and the harm from them as a uniquely modern problem, when harm from social activity has existed as long as social activity has existed. People use chatbots as an escape from the intrisnic harm that is possible from social interaction.

If you look at it that way, you have to pick your poison. Is harm from social activity worse than harm of social isolation when scaled up to the general population. Or rather do the benefits of isolation exceed the benefits of social interaction. In the past this has been rather self balancing.

if chatbots provide a perfect escape from any negative outcomes of interaction, then less and less people develop the necessary skills of empathy and social communication.

Skills relating to interpersonal relations become less and less needed as technology improves, labour becomes more and more compartmentalized and you no longer need social cohesion to get any commodity, you need only interact with the market. I agree that most people are better off talking to real people, but it's nigh impossible to force anything on as large a scale as society when it's not the easiest and most profitable choice for the individual(in this case chatbots are easier and more profitable than IRL social contact, at least for the people that use them.)

You still need to be able to play the "real people" social games to have a seat at higher stakes tables.

How do you feel about humans projecting thoughts and feelings onto toys, plants, animals, characters, and other companionship surrogates?

It’s like taking something unhealthy and turning it up to 11.

The toys, plants, animals, characters will now talk back to them.

I don't think it's too much of a stretch to ask why? A few people have made this point as if it's self evident, but what is unhealthy about it?

Well for starters an AI won’t challenge you and if there is an incentive to make the AI as pleasing as possible then people will become more and more isolated as the relationship with the AI is “better” in terms of not having negative feedback compared to having a real relationship.

Relationships with friends, family, and significant others are full of compromise and learning empathy to maintain that relationship.

But what happens when your primary relationships have no compromise? That you get whatever you want and can be however abusive you want without repercussions. You can see examples of this today without AI and those relationships are generally deemed by society to be “toxic”.

Living in your own private bubble of a AI product written by someone who’s primary goal is engagement won’t give you the tools to function with a group of other living people.

> an AI won’t challenge you

This seems like a shortfall of the current models, and not a limitation of AI companions in general.

> AI is “better” in terms of not having negative feedback compared to having a real relationship

I'd wager that most "real" relationships don't involve nearly as much critique as you're implying. Nobody wants to kill the happy juice their partner generates, whether their partner has skin or circuits.

Plus, there are plenty of other options - such as a councilor - who can provide much better and honest feedback than a personal relationship ever could. One of those compromises we make in a relationship with other meatbags is simply accepting some parts of our partners that we don't like.

Also celebrities.

Remember "Wilson" from Cast Away? That felt realistic to me. A psuedo-relationship in which the partner can actually reply intelligently is just an optional feature.

I met a genius network engineer once who was in a "relationship" with a woman in South America who was obviously fake. He would proudly show me her messages to him that were obviously stolen from some IG or OF model. The texts were pleasant broken English and no real substance. She would go days or weeks without messaging him which clearly took a real, heavy toll on him. They would talk about her moving to the USA someday. Topically of course, never anything concrete. Until one day she finally left him, after ghosting him for weeks. He was devastated. It was sad to watch.

So I posit this to you; Would it have been easier for him to have a fake relationship with a real person, or a real relationship with a fake person?

> So I posit this to you; Would it have been easier for him to have a fake relationship with a real person, or a real relationship with a fake person?

Well, the real relationship eventually broke, forcing him to accept the reality. Also, the fake relationship was probably closer to a real relationship than an AI chatbot would be.

That being said, neither is a good option and this is like wondering whether and SQL injection or buffer overflow is the better vulnerability to have.

I disagree.

One scenario has a human being taking advantage of another human being. One person harming another person covertly. This doesn't give the victim the opportunity to fully understand their situation. This is pure manipulation.

On the other hand, befriending a machine is a choice. You are presented with the facts up front. That this is a machine that imitates companionship. You are not being manipulated or tricked. You are engaging in a form of entertainment, and you can clearly see what you are talking to.

I would rather have a real relationship with a fake thing than a fake relationship with a real person.

That's not the point I was trying to make and it probably was a bad idea to engage with it at all.

An AI girlfriend might be a slight step up, it doesn't really matter. The situation is still far from great either way. In the end, that engineer would be far better of if he could find a real relationship.

It's not a technology that we can "put back in the box", so I suppose we need to find out and deal with the consequences.

Of course we can put it back, or some parts of it, in the box.

Look at things like tetraethyllead and cholorfluorocarbons. Widely known to cause damage, limited upsides. I'm not suggesting we do away with these tools entirely but we can definitely restrict aspects of them for a greater good.

There's quite likely to be a high quality, open source LLM that runs on your mid price GPU in the not too distant future (I give it 5 years max). How do you ban that?

Ban commercialization. You can still get CFCs and leaded gas today if you try hard enough, yet we've drastically reduced their usage.

But with open-source or leaked models, you won't have to try hard at all. Downloading a file is not trying too hard, anyone who is interested will be able to download and run one of those models easily.

So what? We're never going to be able to stop a dedicated user, but if chatbot pseudo-relationships are really an issue we can stop incentivizing their distribution. We don't put ads for fentanyl on busses or let pharma companies give out free samples.

Pharmaceutical companies run promotions and coupons all the time. For prescription medications the samples are just given to physicians who prescribe them.

The difference is that CFCs and leaded gas aren't addictive behaviours.

Not sure if it's better or worse than most of the other parasocial pseudo-relationships people have online. I mean following podcasters, influencers, etc. (and worse).

in some ways it might actually be better, if only because para-social relationships with actual people have led to them (the para-celebrity, for lack of a better word) getting hurt

People develop emotional relationships with non-living things all the time. Phones and cars come to mind.

It certainly isn't. Nothing in the category of reinforcement of self-delusion can possibly be. No "affirmation" can change reality. They can only fit into it or induce living in an hallucinated one.

From the perspective of a single person, every other person is just an agent. The only thing distinguishing delusion from reality is how convincing and sustained the delusion is. If the delusion is indefinite and perfectly convincing throughout, it is reality. What I experience as blue doesn't really exist. The wavelengths exist, but the sensory experience is illusion and highly artificial.

You are using a definition of reality that actually belongs to perceived reality. For that thinking you are proposing, every hallucination is indistinguishable from reality. Is the perfect self-delusion that doesn't need any reality-checks because all you demand is hallucinated-checks to claim is real. But it's not. By the way, notice that thinking is the most fertile ground for lots of ideologies to grow and flourish to the point of anima possession.

The reality can just hit you over the head, the delusion cannot.

Precisely. Reality-checks are not equivalent to hallucination-checks.

What if humans are just pipes and levers?

How often can I talk to someone who has all my same interests, is available 24/7, and will bend to my every will? Could this be a cheap source of oxytocin? dopamine?

The utilitarian makes me think we should embrace it. But its easy for me to say as a married man with 4 kids and living a nice life in the real world.