How is it even an argument? It doesn't illuminate anything, and it's not even clever. It seems like the most facile wrong-headed stab at refutation, by begging the question. As far as I can tell, the argument is, "well you can make this room that manipulates symbols like a computer, and of course it's not conscious, so a computer can't be either"? There are so many problems with this argument I don't even know where to begin.
The fact that he appears to think that changing a "computer" to a "room" has persuasive power just makes it all the more antiquated. As if people can't understand the idea that computers "just" manipulate symbols? Changing it to a "room" adds nothing.
This then devolves into semantics. Can a person locked in a room really come to "understand" chinese culture, for example, if only non-experiential learning were used as data inputs? I think we have to say the answer is yes. I am a chemist. I have never seen an atomic orbital with my bare eyes, yet I can design chemical reactions that work with my understanding of chemistry. Because I have not experienced an atomic orbital does that mean I do not understand? Even, when I set up my first reaction, I did not have any experience, and knew what I was doing only through what could be described as sophisticated analogy. I would say my understading was low, but it was certainly non-zero. Where does one draw the line?
If Searle is right then we should be able to perform a MRI on a blind person while they are talking to someone and spot the point where their brain switches into "symbol manipulation mode" when the conversation subject becomes something visual.
He could have an entire conversation with you in Chinese, and only know that what you said merits the response he gave. He doesn't if he's telling you directions to the bathroom, or how to perform brain surgery.
Searle does try to explain why there's a difference. Although the person in the Chinese Room might be conscious of other things, he has no consciousness of understanding the Chinese text he's manipulating, and will readily verify this, and nothing else in the room is conscious. Chinese speakers are conscious of understanding Chinese.
I mean, agree with it or not, but I think that's a bit stronger than just, making it seem intuitively worse because its a room instead of "a computer"?
I think the important part isn't the swap of "room" for "computer", but instead the swap of "person" for "cpu"?
A better example would be saying something like - does this company have the knowledge to make a particular product? We can say that no individual member of the company does, but the company as a whole does.
Which, well there's a whole series of responses back and forth, with different ideas about what is or is not a good response.
One idea describes a machine where each state of the program is pre-computed, and the computer steps through the states one by one, but in each state, if the next of the pre-computed states was wrong (i.e. would not be the next step of the program, following from the current state), if (e.g.) a switch was flipped, it would cause the program to be computed correctly despite the pre-computed states being wrong, and if the switch is not flipped, then it would continue along the pre-computer states. If the switch is flipped on, or if its flipped off, and all the pre-computer states are correct, the same things happen, and it does not interact with the switch at all. If all the pre-computed states are nonsense, and the switch is flipped on, then it runs the program correctly, despite the pre-computed states being nonsense.
So, suppose that if the pre-computed states are all wrong, and the switch is on, that that counts as conscious. Then, if the pre-computed states are all correct, and the switch is on, would that still be conscious? What if almost all the pre-computed states were wrong, but a few were right? It doesn't seem like there is an obvious cutoff point between "all the pre-computed steps are wrong" and "all the pre-computer steps are right", where there would be a switch between what is conscious. So then, one might conclude that the one where all the pre-computed steps are right, and the switch is on, is just as conscious as the one which has the switch on but all the pre-computed states are wrong.
But then what of the one where all the pre-computed states are right, and the switch is off?
The switch does not interact with the rest of the stuff unless a pre-computed next step would be wrong, so how could it be that when the switch is on, the one with all the pre-computations is conscious, but when it is off, it isn't?
But the one with all the pre-computations correct, and the switch off, is not particularly different from just reading the list of states in a book.
If one grants consciousness to that, why not grant it to e.g. fictional characters that "communicate with" the reader?
One might come up with some sort of thing like, it depends on how it interacts with the world, and it doesn't make sense for it to have pre-computed steps if it is interacting with the world in new ways, that might be a way out. Or one could argue that it really does matter if the switch is flipped one way or the other, and when you flip it back and forth it switches between being actually conscious and being, basically, a p-zombie. And speaking of which you could say, "well what if the same thing is done with brain states being pre-computed?", etc. etc.
I think the Chinese Room problem, while not conclusive, is a useful introduction to these issues?
The states were (probably) produced by computing a conscious mind and recording the result.
Follow the improbability. The behavior has to come from somewhere. That somewhere is probably conscious.
Similarly, authors are conscious, so they know how conscious characters behave.
Pre-computation is another one that seems to obfuscate the actual issue. No, I don't think anyone would think a computer simply reciting a pre-computed conversation had conscious thought going into it; but that same is true for a human being reciting a a conversation they memorized (which wouldn't be that different from reading the conversation in a book). But that's a bit of a strawman, because no one is arguing that lookup table-type programs are conscious (you don't see anyone arguing that Siri is conscious). And the lookup table/precomputations for even a simple conversation would impossibly large (run some numbers, it's most likely larger than the number of atoms in the universe for even tiny conversations).
So I don't see these arguments as bringing up anything useful. They seem more like colorful attempts to purposefully confuse the issue.
Consciousness resides in the minds that created the lookup tables: they were constructed by conscious beings to map queries to meaningful responses.
The lookup tables are the very sophisticated part of Searle's Chinese Room.
The recent emergent semantic vector algebra discovered in human languages by Mikolov's word2vec [Mikolov et al] demonstrate that some of the computational power of language is inherent in the language ( but only meaningfully interpretable by a conscious listener.
Meaning requires Consciousness but language is unexpectedly sophisticated and contains a semantic vector space which can answer simple queries ("What is the Capital of...") and analogise ("King is to What as Man is to Woman") algebraicly. [Mikolov et al]
This inherent semantic vector space is discoverable by context encoding large corpii.
Language is a very sophisticated and ancient tool that allows some reasoning to be performed by simple algebraic transformations inherent to the language.
-
[Mikolov et al] : Efficient Estimation of Word Representations in Vector Space Mikolov, Chen, Corrado, & Dean http://arxiv.org/abs/1301.3781
And because the cpu is the part that does stuff, like the person, then if the system w/ the person doesn't understand chinese, then the computer w/ the cpu doesn't understand chinese.
Because there's nothing to make the cpu more understand-y, only things to make it less understand-y, and otherwise the systems are the same.
Yet here I am, I am the one who is seeing what my eyes see and I am distinct from you. Science still has no idea how that happens, as far as I know.
So how knows, maybe all computer programs are in fact conscious in some way.
Its always distressed me that some people take it seriously as an argument against AI consciousness; it does a better job of illustrating the incoherence of the fuzzy concept of consciousness on which the argument is based.
The Chinese Room argument is that because computers can't "learn", everything has to be taught to them directly, whereas humans are able to take knowledge given and apply it to other situations. While some computers can "learn" enough rules to follow patterns, the argument is that computers can't "jump the track" and that humans can.
Another way to think of it is: a fetus, or a sperm, or ova is not conscious. Some might argue that a newborn isn't really conscious. Somewhere along the line it becomes conscious. How does that happen? Where is the line? We have no idea.
You can't assert that meaning can't arise from "dumb symbol manipulation" without understanding how meaning arises in the former case. We simply don't know enough to make any sort of judgement. The Chinese room argument is trying to make something out of nothing. We don't know.
I think of it this way:
I have two rooms: one has a person who doesn't speak Chinese in it, but they have reference and books that allow them to translate incoming papers into Chinese perfectly.
The second room just someone who speaks Chinese, and can translate anything coming in perfectly.
Searle says that AIs are like person in room one: they don't know Chinese.
I would argue that is the wrong way to look at things. A better comparison is that an AI is like the system of room 1, which does know Chinese, and from observation is indistinguishable from the system of room 2. What's going on inside (a human with Chinese reference books vs a human who knows Chinese) doesn't matter, it's just internal processing.
If it walks like a duck and quacks like a duck, then it's a duck.
If a machine claims to be conscious, and I can't tell it apart from another conscious being, who am I to say it isn't conscious?
The correct question to ask: How is a machine manipulating symbols (that someone says is conscious) different from any other complex physical system? Is New York city's complex sewer system conscious. What about the entire world's sewer & plumbing system?
Does a machine have to compute some special function to be conscious? Does the speed of computation matter? If so who measures the speed? (let us not bring in general relativity as speed of computation can be different for different observers.
Kurzweil et al's definition of consciousness is exactly as silly as Searle saying "My dog has consciousness because I can look at it and conclude that it has consciousness."
IMO Kurzweil's response is actually spot on, although not that hard to come up with. You could make the argument: in your brain, there are just atoms following the laws of physics. The atoms have no choice in the matter, and know nothing about Chinese, or hunger, love, life goals, etc. Your brain is entirely composed of atoms so you can't be conscious.
Obviously "meaning" arises from mindless syntax or mindless physics somewhere in the process. We just don't know where. The Chinese room doesn't bring us any closer to that understanding, and doesn't refute anything.
I think that -- from memory -- this is approximately what Searle believes himself to be saying:
1. Imagine that you're having a conversation with a person in another room, in Chinese. You're writing stuff down on little scraps of paper and getting little scraps of paper back. It's 100% clear to you that this is a real live person you're talking to.
2. Except here's the thing, it isn't. There's actually just this guy in the other room, he doesn't speak read or write Chinese at all. He just has a whole bunch of books and ledgers that contain rules and recorded values that let him transform any input sentence in Chinese into an output sentence in Chinese that is completely indistinguishable from what a real live person who spoke Chinese might say.
3. So, it's ridiculous to imagine that someone could actually simulate consciousness with books and ledgers. There's no way. Since the guy doesn't understand Chinese, he isn't "conscious" in the sense of this example. So we can't describe him as conscious. And the idea that the books are conscious is ridiculous, because they're just information without the guy. So there actually can't be any consciousness there, even though it seems like it. Since consciousness can't be simulated by some books, it's clear that we're just interacting with the illusion of consciousness.
Meanwhile, this is what people like myself hear when he tries to make that argument:
1. Imagine that you're having a conversation with a person in another room, in Chinese. You're writing stuff down on little scraps of paper and getting little scraps of paper back. It's 100% clear to you that this is a real live person you're talking to.
2. Except here's the thing, it isn't. There's actually a system made up of books and ledgers of rules and values in the other room. There's this guy there who doesn't read or write Chinese; he just takes your input sentence in Chinese and applies the rules, noting down values as needed, transforming it until the rules say to send it back to you. That's the sentence that you get back. It's completely indistinguishable from what a real live person who spoke Chinese might say.
3. So, it's ridiculous to imagine that someone could actually simulate consciousness with books and ledgers, but we're doing it for the sake of argument because it's a metaphor that we can hold in our heads. No one would claim that the guy following the rules in the other room is the "conscious" entity that we believe ourselves to be communicating with. And no-one would claim that static information itself is conscious. So either the "conscious" entity must be the system of rules and information as applied and transformed, or else there is no conscious entity involved. If there is no conscious entity involved, and since this is a metaphor, we can substitute "books and ledgers" with "individual nerve cells with synaptic connections" and "potentials of activation", and the conclusion will still hold true; there will still be no consciousness there.
4. However, we feel that there is a consciousness there when we interact with a system of individual nerve cells, synaptically connected with various thresholds of potentiation: even if it's a system smaller by an order of magnitude or so* that the one in our skulls, like our dog has. Thus we must conclude that the "conscious" entity must be the system of rules and information as applied and transformed, or we must conclude that notion of consciousness is ill-founded and inarticulate, that our understanding of consciousness is incomplete, and that our sense of "knowing" that we or another person are conscious is likely an illusion.
*I am fudging on the figure, but essentially we're comparing melon volumes to walnut volumes, as dogs have thick little noggins.
I think I mostly agree with you, but I would argue that if your notion of consciousness is ill-founded and inarticulate, you can't really decide whether it's an illusion either. After all, the subjective experience quite definitely does happen/is real, thus obviously not an illusion, while the interpretation offered for that subjective experience is incoherent, thus there is no way to decide whether it's describing an illusion or not.
Searle would look at that and conclude it had consciousness.
Kurzweil, in summary, asks "You say that a machine manipulating symbols can't have consciousness. Why is this different than consciousness arising neurons manipulating neurotransmitter concentrations?" Searle gives a non-answer: "My dog has consciousness because I can look at it and conclude that it has consciousness."