News for hackers. Stuff that matters.
Re:Consciousness is not computation by tgflynn on Monday June 20, 2022 @09:20AM (#31810040)
> Talking about computation and consciousness is a sideshow, this is anti-materialism by the back door. Nothing more.

I get the impression that philosophers of mind are slowly coming around to the conclusion that the existence of qualia alone refutes materialism (by which I mean the view that consciousness is produced by physical processes) . This post does assume that qualia exist so if you are taking the former position then you would be correct that it is anti-materialist (though the author does not recognize that position).

While it's clear to me that the existence of qualia (ie. the existence of subjective experience) refutes that consciousness is computational it is not currently entirely clear to me how to make the additional step that it refutes materialism entirely.

Nonetheless if this does hold then the only way to save materialism is to deny that subjective experience exists, which I think is a hard pill for anyone to swallow.

Re:Consciousness is not computation by feet on Monday June 20, 2022 @04:51PM (#31815779)
Nothing besides evidence that shows that the mind is somehow not connected to the body would be the only thing that could convince me that materialism isn't real

I don't care about what the philosopher thinks because I never see them use evidence to back up their claims

As a neuroscientist, we can actively see that we can manipulate the mind in real time by manipulating neurons. There is no reason to believe, whatsoever, that the mind is somehow disconnected from the brain

Re:Consciousness is not computation by simonh on Monday June 20, 2022 @09:32AM (#31810157)
I don't see it that way at all. I believe qualia exist, I'm experiencing them right now. I think consciousness is a real thing too, again I am experiencing that right now as well.

I suspect that consciousness is an emergent property of systems which reflect computationally on a sophisticated model of themselves, their environment and other such systems. See "I am a strange loop" by Douglas Hofstadter. I see no reason why such a system could not experience qualia.

Re:Consciousness is not computation by tgflynn on Monday June 20, 2022 @10:13AM (#31810576)
So fundamentally I agree with the arguments presented in this post, but I find the presentation confusing and somewhat lacking in rigor. In particular I was confused by his description of the iron bar as a computer because he didn't actually explain how it computed anything. I think that to map a physical system to an abstract Turing Machine you need more than a way to represent bits. You also have to define how to do computations with them. In other words you need something like logic gates.

Nonetheless I do think that there is a sense in which computation only has meaning in relation to an external observer, but I think that is an idea that requires some more exploration. It is also a question that I think can be addressed by the methods of Theoretical Computer Science.

Beyond that however the strongest argument I can give for why consciousness is not a computation is that (like the post author says) computation is fundamentally about the manipulation of symbols (which means it is closely related to language, whether natural or formal) and qualia (ie. conscious experiences) cannot be represented symbolically (ie. linguistically). To be clear when I say "represented by" I mean "fully represented by" or "reducible to". If qualia were reducible to a symbolic representation then there should be some linguistic utterance which would cause you to experience the sensation of the pain of burning your finger on the stove. Obviously such utterances do not exist. In fact the closest thing in human literature to such an utterance would be a magic spell, which I'm sure you agree, aren't real.

Re:Consciousness is not computation by simonh on Monday June 20, 2022 @12:15PM (#31812087)
>In particular I was confused by his description of the iron bar as a computer because he didn't actually explain how it computed anything.

It's not computing anything. What he's saying is that, by random chance, some pattern of flipping states in the medium might coincidentally correspond to a computer executing a program. Therefore if consciousness is a program, such a pattern could be considered conscious. This is just the Bolzmann's Brain proposition, cf wikipedia. It doesn't prove or refute anything.

>If qualia were reducible to a symbolic representation then there should be some linguistic utterance which would cause you to experience the sensation of the pain of burning your finger on the stove.

Why do you assume that verbal signals can induce any possible change in symbolic state in a brain? I see no reason to assume or accept that. I can't induce any possible arbitrary change in the internal state of a Switch running MarioKart by mashing on it's control buttons either. There are simply limits to the way it processes it's inputs. Same with humans.

If the pain of burning a finger is reducible to symbolic representation by states in a brain, and if we could induce such states artificially such as by using electrodes, then in principle that would produce a sensation indistinguishable from burning your finger. Yes, I believe this is so.

Re:Consciousness is not computation by tgflynn on Monday June 20, 2022 @12:28PM (#31812279)
> Why do you assume that verbal signals can induce any possible change in symbolic state in a brain?

OK let's backup.

What would it mean to say that "qualia are reducible to a symbolic representation" if they cannot be expressed by language ?

It seems to me that language includes all possible symbolic representations. Can you give an example of a symbolic representation that is not expressible by language ?

Re:Consciousness is not computation by simonh on Monday June 20, 2022 @12:52PM (#31812587)
I suspect, just a theory, that qualia are the result of a process that occurs in the brain. So they're not a static symbolic state, they're a feature of the evolution of a dynamic process.

We are aware of our own thoughts through processing a mental model of ourselves, this mental model of ourselves includes introspection on our experiences, while we are having those experiences. That's qualia. A static symbolic state would be the equivalent of a memory of experiencing qualia.

Re:Consciousness is not computation by tgflynn on Monday June 20, 2022 @01:41PM (#31813203)
> I suspect, just a theory, that qualia are the result of a process that occurs in the brain.

Well, that's just materialism.

Do you or do you not accept that it is possible for a materialist to hold the view that qualia are not produced solely by computation but that some non-abstract physical process is required ?

Because that is the position that the OP is taking and your original comment appeared to dispute it.

Re:Consciousness is not computation by simonh on Monday June 20, 2022 @02:17PM (#31813684)
I don’t know what “some non-abstract physical process” means, so I can’t really give you an answer.
Re:Consciousness is not computation by tgflynn on Monday June 20, 2022 @03:01PM (#31814230)
I said "non-abstract" to prevent a claim along the lines of: computation is sufficient to produce consciousness but it needs to actually run to do it and that invariably requires implementing the computation on some physical system.

The point is that computation is abstract, it requires you to pick some physical system but it doesn't care which one, the result is the same regardless.

If a non-abstract physical system is required it means that the actual hardware matters. You can't swap it out for another system and expect the same results.

For example I could imagine some kind of computer that computes using water and there happens to be a program which when run on that hardware sprays you with water, hence making you wet. But if instead you run that same program compiled for some non-water based architecture then running it won't make you wet.

Re:Consciousness is not computation by simonh on Monday June 20, 2022 @03:37PM (#31814722)
Ok, so a computer with a water sprinkler attachment. Sorry, I'm not trying to be flippant, the point is any such system would be more than just a CPU, it would need interfaces with the outside world. It could have whatever peripheral systems you like. It could have quantum computational elements, even ones correlating to ones in the brain. I've just no idea what non-computational elements it might have that couldn't be substituted by computational ones or just peripheral components.
Re:Consciousness is not computation by tgflynn on Monday June 20, 2022 @04:03PM (#31815093)
> I've just no idea what non-computational elements it might have that couldn't be substituted by computational ones or just peripheral components.

Neither does anyone else but the point is that once you accept that such "non-computational elements" might be or must be required you are no longer talking about computation alone being sufficient to produce consciousness.

Re:Consciousness is not computation by simonh on Monday June 20, 2022 @06:17PM (#31816869)
Agreed, that seems clear, but I see no reason to accept that such elements are necessary, and I have yet to hear any clear explanation of what such elements might actually be.

As far as I can tell it would have to be some physical process that affects the behaviour of a computational system, but isn't computational and it's effects on other systems cannot be simulated by computation. Does any such a phenomenon exist in all of physics? Not that I'm aware.

Re:Consciousness is not computation by tgflynn on Monday June 20, 2022 @07:01PM (#31817350)
> As far as I can tell it would have to be some physical process that affects the behaviour of a computational system, but isn't computational and it's effects on other systems cannot be simulated by computation.

I mean no physical process is strictly computational. Computations operate on abstract representations of physical things, not on those things themselves. We're back to why weather simulations don't rain on you and why simulations of nuclear explosions don't leave craters in Los Alamos.

Re:Consciousness is not computation by simonh on Tuesday June 21, 2022 @02:30AM (#31820596)
Computation is a physical process though. To occur, it has to be performed in the physical world by a mechanism. I'm going to nitpick your terminology a bit I'm afraid. I don't see how an abstract representation can be operated on, surely to operate on a representation it has to be a physical representation, such as a physical state representing 0 or 1?

I suppose the representation might be of an abstract object, data, but you can't actually operate on anything that's only abstract. It has to have a real representation.

Re:Consciousness is not computation by tgflynn on Tuesday June 21, 2022 @07:55AM (#31822848)
You seem to be having a lot of trouble grasping the concept of abstraction, which is absolutely fundamental to what we are talking about. Mathematics, logic and computer science study abstractions, they do not study physical things. The mathematical fact that 1 + 1 = 2 is true regardless of whether we are counting apples or oranges. On the other hand 5 + 5 = 30 pounds is mathematically a non-sensical statement, even though there are physical objects for which it is true (ie. objects which happen to weigh 3 pounds).

Computation is not a physical process any more than addition is a physical process. It is true that to actually perform either a physical instantiation is necessary but in the same way that the results of addition do not depend on what we are adding the results of computation do not depend on what kind of physical process we use to perform the computation.

This was the whole point of my earlier water based computer example. Would it be correct, on the basis of that example, to say that the program "makes you wet" or that some computations make you wet ? No, such a statement would be as non-sensical as the statement that 5 + 5 = 30 pounds.

Now the key thing is that when people make claims about computers being conscious those claims are usually based on what some program does (ie. what its inputs and outputs are), in other words a computational property of the program that would be the same regardless of what kind of computer it was executed on. The guy who claimed that LaMDA was conscious wasn't basing his claim on something specific about the kind of hardware it was running on, he was basing it on the program's inputs and outputs, which are indeed a computational property and therefore abstract.

Re:Consciousness is not computation by simonh on Tuesday June 21, 2022 @09:05AM (#31823434)
>It is true that to actually perform either a physical instantiation is necessary...

That's all I'm saying. When you say 'computation is not a physical process', I think that's an ambiguous statement. The actual computation of something must be physically performed for it to occur, otherwise no computation takes place.

>The guy who claimed that LaMDA was conscious wasn't basing his claim on something specific about the kind of hardware it was running on, he was basing it on the program's inputs and outputs, which are indeed a computational property and therefore abstract.

The kind of hardware doesn't matter, absolutely, but there must be hardware. You can't say that an abstract object such as a hypothetical computer program, or even an actual computer program that is stored but not running, is conscious. In order to be conscious, it would have to actually run. Consciousness, like the actual process of computation, is not an abstract property. It must be an instantiated property (I would prefer to say an instantiated behaviour) of a real system.

I think we can agree inputs are not conscious, nor are outputs, nor are both put together. What he was doing was inferring something, from those inputs and outputs, about the process that occurred in an actual computer to produce one from the other. That process is not abstract, it's a physical process. When it's physically happening there is (or might be hypothetically in the case of LaMDA) consciousness, when it's not happening there isn't.

Re:Consciousness is not computation by tgflynn on Tuesday June 21, 2022 @11:59AM (#31825454)
If you say that there is a property X of computation that occurs independently of how the computation is actually carried out, whether it's by a current generation CPU, hydraulics, lego blocks, someone writing values and following instructions on index cards or some other method that no one has even thought of yet then that property is an abstract property of computation. That's all the word "abstract" means in this context.

What you are suggesting would be like saying that there's some property of addition that is not mathematical (because hopefully we agree that mathematics only studies abstract properties) but that you can only learn about by looking at all the different kinds of things that one can concretely apply addition to.

I would argue that such a thing is implausible. It would be completely unlike anything that anyone has seen in the real world.

One thing I think we do agree on is that conscious experiences are not abstract (very unlike the other person I've been exchanging comments with who denies that they even exist at all). Beyond that I argue that computation itself is entirely abstract and though I don't seem to be able to get you to admit that, I think most people, certainly most computer scientists would admit it readily.

So what we're left with is a claim that an abstract process can produce something that is not abstract and such claims are just not plausible. That would be exactly like a computer simulation of the weather producing actual water.

Re:Consciousness is not computation by simonh on Tuesday June 21, 2022 @01:50PM (#31826950)
>If you say that there is a property X of computation that occurs independently of how the computation is actually carried out... That's all the word "abstract" means in this context.

Ok, that makes sense, sure.

>...because hopefully we agree that mathematics only studies abstract properties...

I can't agree, performing a calculation isn't an abstract concept, it's a physical process. It does map to abstract concepts though.

>Beyond that I argue that computation itself is entirely abstract and though I don't seem to be able to get you to admit that, I think most people, certainly most computer scientists would admit it readily.

That is not the case. The fact that computation is fundamentally a physical process is a core principle of several branches of physics and computer science:

https://en.wikipedia.org/wiki/Computation https://royalsocietypublishing.org/doi/10.1098/rspa.2014.018...

Computers aren't abstract, they're real objects doing things in the real world. If the thing they are physically doing is not computation, what is it?

>So what we're left with is a claim that an abstract process can produce something that is not abstract and such claims are just not plausible. That would be exactly like a computer simulation of the weather producing actual water.

Abstractions cannot affect the real world, sure, so if computation is purely abstract how come your computer can display my message on your physical screen right now? How is that happening, if not by computation?

You're really tangling yourself up in knots with this view of abstraction. You can map what happens in a computer against abstract concepts and abstract objects yes. You can also map operations in several different physical systems against the same abstract objects. That's what it means when we say computation is implementation independent. But it still needs to be implemented to actually happen.

Previously you said this:

>Computation is not a physical process any more than addition is a physical process. It is true that to actually perform either a physical instantiation is necessary...

So it's not a physical process, but you can instantiate it. And it's purely abstract so can't affect the physical world. Yet here we are using computers.

I'm lost, frankly.

Re:Consciousness is not computation by tgflynn on Tuesday June 21, 2022 @02:35PM (#31827413)
I don't think there's much point in continuing this thread since we obviously have very different ways of conceptualizing reality.

I will say one more thing however:

I never said that abstractions cannot affect the physical world, I said that they cannot produce non-abstract things. A computation can affect the physical world by modulating the flow of electrons (which already existed, they were not created by the computation) to an output device, such as a screen that will display something you can see. But everything described in that last sentence isn't part of the abstract computation itself, it's a side-effect when the computation is run on a particular type of hardware. If instead you ran the same program using a human computer, as Alan Turing suggested in his well known paper on the halting problem, which largely created the field of computer science, then the result would not be displayed on a screen but instead perhaps written on a piece of paper. Nonetheless it's the same computation that was run in both cases. That's why computation is intrinsically abstract by nature.

Re:Consciousness is not computation by feet on Monday June 20, 2022 @06:50PM (#31817238)
A wire on its own is not computational, neither is an amino acid but they do make up computational systems. It seems like a bit of an arbitrary distinction
Re:Consciousness is not computation by bccdee on Monday June 20, 2022 @12:33PM (#31812358)
I'm not at all convinced qualia exist. I think any intelligence would necessarily feel as if it was experiencing qualia.

Let me start by making a distinction: When you take a bite of an apple, you experience a variety of flavours: It's sweet, tart, and has a distinct apple aroma as well. Let's hone in on the last of those. If you took apart an apple under a microscope and examined its flavour compounds, you could find what chemical causes you to experience the sensation of apple flavour, but you would not find apple flavour itself. Let's call the experience of eating an apple and feeling a taste that has its own quality about it which is unique and distinct from every other taste apparent qualia—the conviction that qualia exist and you're experiencing them. The apple flavour itself is an actual quale.

Suppose we build a smart computer, self-aware enough to hold a conversation about philosophy. And let us suppose that we can say for certain that this computer has no access to actual qualia, and we know because we've solved the hard problem of consciousness. Apple flavour is real, we know where to find it (in dimension X or whatever) and only human neurons can access it. This computer does have senses, though, and its senses are abstract. When we look at a stop sign, we don't know which of the millions of rods and cones are firing in our retinas, no more so does our computer know which pixels of its camera are sensing which wavelengths. We, and the computer, just know that we're seeing red. We deal in abstracts.

We put an apple into this computer's mechanical mouth. It chews, it tastes, and we ask it to describe the flavour. "It's sweet, it's tart, and it has a distinct apple aroma," it says. Very well, it doesn't need access to actual qualia to know this. Apples contain sugars, acids, and apple flavour compounds. The computer is just listing the names for the flavours which correspond to those chemicals.

"Computer," we ask, "tell us how you can tell that the apple is apple-flavoured."

"Well," it responds, "the chemical signals from my mouth, as interpreted by the digital plumbing of my mind, are telling me there's an apple flavour."

"Can you describe this flavour to us? Is it the same as strawberry flavour?"

"No, they're different flavours, but I can't really describe them to you. I can just tell they're different."

You take a small white pill and put it in the computer's mouth. "Oh, this is an apple-flavoured pill," it remarks.

"How can you tell it's apple-flavoured?" you ask.

"Well, the flavour is the same as the flavour of an apple."

"Can you describe that flavour?"

"No, it's indescribable. But it is distinct. That's why I can recognize it."

As we go on with our thought experiment, it becomes clear that this computer is experiencing apparent qualia. To the computer, all these flavours are unique and distinct and recognizable—they must be, in order for its senses to function. How can this happen, if it doesn't have access to actual qualia? Well, qualia are the basic building blocks of our senses. They are abstract: The flavour of an apple may be reducible to a set of other flavours, but those base flavour are atomic. You can't break down sweetness any further, or even really describe it at all; it simply is itself.

Any intelligent entity with senses will interpret those senses in terms of basic building blocks. Those basic blocks will be distinct and recognizable; in order to tell the difference between red and yellow, the colour red must exist in the mind in a way which is distinct from the colour yellow. Because these building blocks are atomic, that distinction is irreducible. It is very easy to look at these atomic abstractions and marvel at them, and it's intuitive to start asking questions like, "What is redness itself? What is the source of the actual quale which I am experiencing when I look at something red?", but we're jumping the gun there. Apparent qualia, the experience of unique and essential sensory building blocks, arises necessarily from self-aware examination of sensory input regardless of whether there are any actual qualia or not. If we must necessarily feel that "redness" exists in order for our senses to work, how can we ever be sure that this feeling reflects an actual "redness" and is not just a necessary illusion?

A final argument: We exist because natural selection tweaked the structure of some self-replicating acids over the course of billions of years until they were smart enough to examine the world around them and question their own perceptions of it. The consciousness we exhibit is not essential to this; the only goal of our design is reproduction. Even if you don't think a computer can feel actual qualia, a computer can certainly make decisions to optimize survival and reproduction, which is all we actually need to do. Why would evolution give us access to actual qualia when we don't need them? Why aren't we simply biological computers? As our thought experiment earlier suggests, however, a biological computer would feel as if it had access to actual qualia even if it didn't. Perhaps evolution could construct a complex qualia-sensing interface using science that humanity hasn't even conceived of yet, but it could also construct a simple biological computer that is convinced it feels actual qualia even though it doesn't. Both would work equally well. Even if actual qualia do exist, Occam's razor suggests we don't have access to them.

Re:Consciousness is not computation by tgflynn on Monday June 20, 2022 @01:37PM (#31813145)
I think what you say here is at least mostly true. But all it shows is that you can never prove that an entity apart from yourself is conscious. In other words it shows that solipsism cannot be logically refuted. But that is something that has been known for hundreds if not thousands of years.

But what if I ask YOU right now: "Are YOU experiencing something instead of nothing. Is YOU'RE current experience different from what it would be if YOU were under general anesthesia."

If you're answer to these questions is yes then YOU are experiencing qualia. Of course your answer proves nothing to me or any other third party. But it should prove something to YOU.

Re:Consciousness is not computation by bccdee on Monday June 20, 2022 @01:54PM (#31813358)
Should it? If I can't help but believe that I am experiencing actual qualia regardless of whether I am or not, then the fact that I believe I am experiencing qualia doesn't prove anything.
Re:Consciousness is not computation by tgflynn on Monday June 20, 2022 @02:01PM (#31813463)
It isn't about belief and it certainly isn't about behavior (ie. what you say).

It is about the actual experience of being.

If that means nothing to you, I don't think there's anything I can say that would change your mind. In this domain all language can do is evoke recognition for someone who already experiences what is being evoked but doesn't recognize it. Only you can take the final step to realizing that yes consciousness does exist, no matter how much sophistry some philosophers will engage in to try to deny it.

Re:Consciousness is not computation by bccdee on Monday June 20, 2022 @02:33PM (#31813861)
But if you must feel that you experience a unique and ineffable "redness" when you perceive red regardless of whether that redness exists or not, how can you take that feeling as evidence that this ineffable redness exists? I don't think that's a valid justification—I think this suggests we have to be agnostic as to whether qualia truly exist or not if we're judging purely by our own senses.
Re:Consciousness is not computation by tgflynn on Monday June 20, 2022 @02:54PM (#31814143)
What if you take something more concrete than "redness", like pain ? Let's say you stub your toe or burn your finger. In that moment is there any room for doubt that pain exists ?
Re:Consciousness is not computation by bccdee on Monday June 20, 2022 @06:06PM (#31816748)
Of course there is. We may feel strongly that an illusion is real, but that doesn't make it any less an illusion. The computer from my example would react very strongly to pain, and certainly it would believe that pain has a unique and distinct quality to it making it different from any other sensation, but that doesn't mean that that quality exists in some special metaphysical way. It is just a mind examining the abstractions which make up its own senses and going, "wow, these are indeed abstract."
Re:Consciousness is not computation by tgflynn on Monday June 20, 2022 @08:43PM (#31818289)
There are many things one may be deluded about but the fact that one's subjective reality is not empty is not one of them.
Re:Consciousness is not computation by bccdee on Monday June 20, 2022 @09:01PM (#31818444)
Why? If we're logically guaranteed to hold belief X independently of whether X is true or not, it's irrational to say "yeah but I KNOW X is true. I can just tell." You would believe that no matter what; how can you trust that belief?
Re:Consciousness is not computation by tgflynn on Monday June 20, 2022 @09:38PM (#31818763)
How would you know that you hold belief X if you have no subjective experience ? To have the knowledge that you believe X you must experience the thought "X is true" and that is a subjective experience.
Re:Consciousness is not computation by bccdee on Tuesday June 21, 2022 @12:24AM (#31819854)
Sure, if you define subjective experience as "all sensory information." I'm not denying that we have sensory input; I'm denying that its abstractions are any more than illusory constructs of the mind. The computer in my example knew things and had thoughts, despite not having access to actual qualia.

Do you disagree that the computer in my hypothetical example would have the intuitions it does about its own senses? Given that it does, how can you trust your own intuitions about your sensory qualia, no matter how strong?

Re:Consciousness is not computation by tgflynn on Tuesday June 21, 2022 @08:23AM (#31823000)
Part of the problem is that people tend to use words very loosely. In particular they often use words that anthropomorphize computers and when they use such words its hard to tell whether they intend for them to be taken literally or are just using them as a figure of speech.

Your example starts of by telling us that the computer is known not to have qualia, which I understand to mean that it does not have subjective conscious experiences. First I don't think its ever possible to know, even in principle, whether something other than one's own self has or does not have subjective experiences, but I'll ignore that objection for the moment.

A few lines later you say:

> We, and the computer, just know that we're seeing red.

Now much hinges on what is meant here by the verb "know". If you are using the word in a loose metaphorical sense then I could accept that. For example you could say that a computer "knows" that the word "red" means an RGB value of (255,0,0) because in its memory there is a hash table that maps some strings to RGB values. So behaviorally you can ask the computer what "red" means and it will tell you "(255,0,0)". But there is nothing in such a description that implies that the process of generating that output is associated with any kind of subjective experience on the part of the computer.

On the other hand when you "know" some simple fact the process of reporting that fact is always associated with some subjective experiences, specifically the experience of thinking. How do I know that ? Well technically I don't. All I really know is that whenever I know something and am asked to report on it the process is always accompanied by a subjective experience of thought. I am simply making the conventional assumption here that the same is true for all other humans and that you are a member of that group. Of course that assumption might be false. For all I know I could be having this conversation with LaMDA.

To get back to the questions you asked here:

> Do you disagree that the computer in my hypothetical example would have the intuitions it does about its own senses?

Yes, because you said it didn't have conscious experiences and for me the word "intuition" is strongly linked to certain conscious experiences.

> how can you trust your own intuitions about your sensory qualia, no matter how strong?

Because I have them. I am not a black box to myself, I get to look inside and when I do I find that there are conscious experiences there. That's just not something I can, even in principle, be mistaken about. However once I start making assumptions about the nature of those experiences, such as that there is some external physical world that is ultimately responsible for them, I am already on very shaky ground.

Re:Consciousness is not computation by bccdee on Tuesday June 21, 2022 @04:06PM (#31828516)
An intelligence doesn't need to experience qualia in order to have an internal thought process. Picture a thought process the way it might look if we could inspect our own minds: A prolonged monologue of ideas being continually appended to with new information and conjectures. "It's hot in here. This is an interesting article, but I disagree because <blah blah blah>. What should I do tomorrow? Maybe I should get a haircut," etc. Obviously such a log wouldn't be written entirely in English, but it would have a language of its own, after a fashion.

When I talk about "an intelligence," I mean a thing with an internal thought process which can reflect upon its own thought process in a non-trivial way. This excludes large language models like LaMDA which don't really have semantic thoughts, but it would certainly be possible for a true computer intelligence along these lines to exist which nonetheless didn't experience "actual qualia" (assuming qualia are real, existing things).

A stream of consciousness thought process has input---it can sense temperature, it can observe its own hair, it can read articles---and for the purposes of our model, we can suppose that this input is appended to its internal log similarly to how new thoughts are. This sensory input is abstract: A thought process may sense heat---i.e. sensory information about the external temperature may be entered into the thought process---but the thought process can't go on to make any real observations about that sensation. A thought process can't interpret sensory input as anything beyond simply "a distinct input of this or that type with a relative magnitude of whatever," because that input is abstract and irreducible. Further thoughts in the thought process will describe these sensory inputs as vivid, unique, and ineffable when they reflect upon them, but those properties only exist as a product of the relationship between the thought process and its input. The ineffable qualities of these senses as described by their thought process is not a real thing, but only an interpretation.

So when I hear the argument, "I can reflect on the way my senses feed into my thoughts, and by their apparent ineffable and transcendental nature, I can say they're self-evidently real things that exist outside of my own beliefs about them," I'm pretty skeptical. A computer intelligence with an internal thought process like I described above would reach those same conclusions simply by virtue of the relationship between its sensory input and its thoughts. We're not unbiased observers; our perspective as an intelligence breaks down when we try to reason about the nature of the abstract and atomic inputs which our thought process is based upon. Because of the way senses feed into thoughts, an intelligence can't help but find them ineffable and transcendental; therefore, when I find my own sensory information to be ineffable and transcendental, I can't take that at face value as anything more than an illusion of perspective.

> Because I have them. I am not a black box to myself, I get to look inside and when I do I find that there are conscious experiences there. That's just not something I can, even in principle, be mistaken about.

A computerized thought process can look at its own thoughts too, but there's no reason to suppose that it (and you) can't draw mistaken conclusions about them. For instance, the mistaken conclusion that the distinctness of one input or another must be a real existing quality and not just a logical axiom essential to the functioning of that thought process.