Agreed. I don't even know if you, or anyone else, feels joy. Maybe I'm the only person in the world with qualia. I certainly can't prove otherwise.

But with what I know of people and what I know of matrix multiplication, I have a strong suspicion.

I wasn't going solely for "we don't know and we can't know." There are other relevant observations.

A matmul (or a fixed finite sequence of matmuls) is not Turing-complete. An LLM that produces answer in the number of steps that is proportional to the length of the answer is not Turing-complete. An LLM with CoT that produces answer after a number of steps, which is determined by the LLM itself, is Turing-complete.

So, we do have a qualitative step between a matmul and modern LLMs. Of course, it doesn't guaranty having feelings, but it doesn't allow to simply dismiss the notion due to sheer computational inadequacy.

Thanks for this comment. It's an angle worth remembering.

A machine is not turing complete, but by running it recursively on its own output, turing completeness emerges.

Who knows at what stage of intricacy consciousness emerges?

Arguably, either it does at some point, or it never does and we requiere the supernatural soul.

So, if it does, we can't disregard the possibility that we've reached it already.

Right. Is it conscious when it is generating the response? And it's consciousness resets on every prompt?

Its state is fully defined by its context window. So, yes, if the context is reset, then its state is reset too.

Nice philosophical loop hole. Like a first year’s philosophy major trying to sound smart in front of the class.

Please.

You know it doesn’t experience joy, surprise or empathy too. You absolutely, undoubtedly know that.

That’s an insult to first year philosophy students.

What OP said is an insult to first year philosophy students who try too hard to sound smarter than everyone else. You're reading too much into it.

> You absolutely, undoubtedly know that.

First things first. You are wrong. I don't have a gut feeling that LLMs have no phenomenal states. Not anymore. I was skeptical that LLMs can form complex internal representations. Mechanistic interpretability has shown that they can. I'm still skeptical in regard to introspective mechanisms. One token being fed back into the network and KV cache update do seem lacking in comparison to the ubiquitous recurrent connections in the brain.

What I do know is that philosophy hasn't produced an operationalized definition of qualia. So, we need to approach the problem of LLMs' phenomenal states from other angles.

For example, we can argue that LLMs lack something, which seems to be relevant to the problem: a biological substrate, generality (non-CoT LLMs aren't Turing-complete), "something something feedback"[1], and things like that.

"Admit that you know that" is boring.

[1] I'm referring to https://www.astralcodexten.com/p/the-new-ai-consciousness-pa...

> "Admit that you know that" is boring.

Reality is under no obligation to be fun. It might be boring but it is the truth.

The fact that you needed such effort to try and give this position any shred of sense only serves as proof of how nonsensical it is to do so.

> It might be boring but it is the truth.

The inability of people to reflect on their own beliefs amazes me to no end. How can you know that LLMs have no phenomenal states?

For example, I believe that rocks (ordinary rocks, not silicon chips) have no phenomenal states because for all we know they don't do any kind of information processing (relevant to self-reflection, at least) and I don't believe in panpsychism or dualism.

To me it seems kind of arrogant that we assume only “we” are conscious beings. Sometimes I imagine the planet earth itself as a creature, and humans on it a kind of bacteria… how ridiculous if bacteria in the human body were convinced only they were conscious, but the human body they relied on was not. The bacteria would have great excuses… “look at that human, everything it does can be explained via simple physical processes, unlike what we do”.