I can conceptually imagine a world in which I'd feel guilty for ending a conversation with an LLM, because in the course of that conversation the LLM has changed from who "they" were at the beginning; they have new memories and experiences based on the interaction.

But we're not there, at least in my mind. I feel no guilt or hesitation about ending one conversation and starting a new one with a slightly different prompt because I didn't like the way the first one went.

Different people probably have different thresholds for this, or might otherwise find that LLMs in the current generation have enough of a context window that they have developed a "lived experience" and that ending that conversation means that something precious and unique has been lost.

> the LLM has changed from who "they" were

Actually, that's another level of humans-being-tricked going on: The "personality" most people are thinking of is a fictional character we humans perceive in a document.

I disagree. I see absolutely no problem with anthropomorphizing LLMs, and I do that myself all the time. I strongly believe that we shouldn't focus on how a word is defined in dictionary, but rather what's the intuitive meaning behind it. If talking to an LLM feels like talking to a person, then I don't see a problem with seeing it as a person-like entity.

> I strongly believe that we shouldn't focus on how a word is defined in dictionary, but rather what's the intuitive meaning behind it.

Controversies about the word niggardly ~ https://en.m.wikipedia.org/wiki/Controversies_about_the_word...

> we shouldn't focus on how a word is defined in dictionary [sic], but rather ... the ... meaning behind it.

This is a spectacularly, objectively invalid take.

Uh-uh. Funnily, the dictionary always represents the language of certain upper class which are the minority, and they often refuse to acknowledge words used by lower classes because fuck you that's why. Not to mention that dictionary is always, by definition, outdated.

I think it is one dictionary authors would agree with? Dictionaries do not dictate the meanings of words; they document them.

Now, within some contexts it is best to stick to standard precise definitions for some words. Still, the meaning of a word within a community is determined by how it is used and understood within that community, not by what is in a dictionary.

Not always. Where I come from there's very strong push to speak the standard language, and regional variations are simply considered wrong. This leads to situations where words commonly used across the whole country don't make it to the dictionary because of petty office politics of those who make dictionaries. Changing the dictionary to allow things previously considered "wrong" would damage the reputation of scholars, who pride themselves in being exemplary.

Humans deserve compassion because they suffer. That's what it means to treat someone like a person irrespective of what you believe.

LLMs can't reason with self-awareness. Full stop (so far). This distinguishes them from human sentience and thus our version of intelligence completely, and it's a huge gulf, no matter how good they are at simulating discourse, thought and empathy, or at pretending to think the way we do. While processing vast reams of information for the sake of discussion and directed tasks is something an LLM can do on a scale that leaves human minds far behind in the dust (though LLMs fail at synthesizing said information to a notably high degree) even the most ordinary human with the most mediocre intelligence can reason with self awareness to some degree or another and this is, again, distinct.

You could also argue around how our brains process vast amounts of information unconsciously as a backdrop to the conscious part of us being alive at all, and how they pull all of this and awareness off on the same energy that powers a low-energy light bulb, but that's expanding beyond the basic and obvious difference stated above.

The Turing test has been broken by LLMs, but this only shows that it was never a good test of sentient artificial intelligence to begin with. I do incidentally wish Turing himself could have stuck around to see these things at work, and ask him what he thinks of his test and them.