NPCs don't quack like people. There's a reason no one was seriously having this argument(s) with the likes of Eliza(which is a step up from pre-recorded response NPCs). This goes beyond superficial production of text.

In principle it's possible to track and reproduce all the neuron/synapse communications that happen in your brain in relation to any arbitrary input. Where's the subjective experience there?

As far as anyone knows, qualia is emergent. Individual neurons or synapses don't have any understanding of anything. Subjective experience comes from all those units working together.

A single ant doesn't display the intelligence/complexity of its colony.

> In principle it's possible to track and reproduce all the neuron/synapse communications that happen in your brain in relation to any arbitrary input

This hypothetical is doing a lot of heavy lifting. The reality is that our understanding of the brain is exceedingly crude, a perceptron in software is not even close to the same thing as a neuron in the brain. A plane can never become a bird even though it is modeled after one.

> A single ant doesn't display the intelligence/complexity of its colony.

So do you think that an ant colony has subjective experience?

>This hypothetical is doing a lot of heavy lifting.

Not really. Point is that unless you believe in mysticism/magic/spiritualism behind consciousness in the brain then it doesn't matter because the only difference is degree of understanding not implausibility.

>So do you think that an ant colony has subjective experience? Sure it could. I don't know for sure. Nobody does. Humans don't see subjective experience. I can't prove that you have one. I'm just assuming you do. Same as you for anyone else.

> Sure it could. I don't know for sure. Nobody does.

Yet your comments don't reflect that kind of skepticism when referring to LLMs. However, if you're fine with the panpyschist conclusions that follow from your reasoning, we don't have much to disagree on.

The problem, as usual, is in how exactly words like "subjective", "experience", and "conscious" are defined. If you have a particular detailed definition for "quacking" and a test for it and an entity passes the test, then by that definition, it quacks. Words like "conscious" are notoriously nebulous and slippery, and so it's better to use the particular attributes in question.

If we could all agree on a particular definition for "consciousness" and a running video game fits that definition, then, like it or not, it's conscious. But it does not mean that it must now also have a slew of other attributes just because conscious humans have them.

(edit: 10 hours later and ninja'ed by 1 minute, of course)

Subjective experience has a pretty clear meaning. Cogito ergo sum. Unless you're a panpsychist, it is assumed that things like people have subjective experience and things like rocks don't have it. We don't have a causal explanation for subjective experience, but there's absolutely no reason to believe that computer programs have them any more than rocks. In fact, a rock is actually more likely to have subjective experience than a LLM since a rock at least has a temporal identity, LLMs represent a process not an entity.

I have no problem saying a tape recorder has subjective experience: it's subjective (the recordings it makes are its own), and it experiences its input when its been commanded to record, and can report that experience when commanded to play it back. Note this does not mean I think that a tape recorder can do anything else we humans can do.

What is being experienced (recorded) is not the process of experiencing per se. Most people don't separate the two, which leads to endless disagreements.

But yes, a computer program by itself can't have subjective experience. Nor a rock. At least until it weathers, gets turned into a silicon, and into a computer to run that program. Then it's all information processing, for which subjective experience is trivial.