You're describing the systems reply to the Chinese Room Argument[1] and I agree with you. To go further, what else could possibly generate consciousness? A soul? Some phlogiston that breathes fire into beings? I legitimately don't understand what the alternative is to physicalism without positing some completely unsubstantiated Other Thing that also gets us no closer to understanding consciousness.

[1] https://en.wikipedia.org/wiki/Chinese_room#Systems_and_virtu...

I agree that other mediums (like artificial neurons) would likely still be conscious, but an abstract calculation on a bunch of pieces of papers, possibly spread over years and across multiple people performing the calculations? That seems odd to me. Definitely still "intelligent", but should I feel bad about ripping up those pieces of paper? Should I feel like I'm killing someone by erasing calculations off of the paper or introducing a mathematical mistake somewhere in the middle?

What does morality have to do with the nature of what is? How you feel about snuffing out some form of consciousness is irrelevant to whether it exists or not.

> should I feel bad about ripping up those pieces of paper?

Do you feel bad every time you have a beer and potentially kill off a few brain cells?

The idea that scribbles on a paper would be conscious is really hard to believe. It also sounds like it's own form of dualism that's functional instead of substance-based. I wouldn't consider it a good physical theory of consciousness.

>The idea that scribbles on a paper would be conscious is really hard to believe.

But it's not that the scribbles on paper are conscious; scribbles on paper have no causal powers for example. The scribbles on paper are the volatile storage of this causal chain: the causal cascade flowing through the actions of the person reading the symbols, manipulating them, then writing out new symbols. The bait-and-switch is that when you adjust the thought experiment, the focus shifts from the unified casual process performing certain computations to the inert substrates of the paper (or in the case of the Chinese room, the man performing the computations). But no matter the medium of computation, the causal chains are still instantiated and so there's no reason to think this process is not conscious.

Fair enough, but what makes the instantiation of causal cascade conscious? I see no reason to suppose it would feel pain or see color. What makes it so?

You're looking for a hard cut off where none exists. Most likely everything is conscious on many different levels and scopes of capability.

So the case which makes people really uncomfortable - are cows conscious (because we eat them) has the very unsatisfying answer of "yes, not in the same way we are most likely, but in some way".

There is indeed a lot about the workings of minds that neither I nor anyone else knows, but I do not mistake my lack of knowledge for strong evidence that certain things cannot be so. That would be an example of what Dennett calls the philosopher's syndrome: mistaking a failure of the imagination for an insight into necessity.

It would be like a pre-Archimedean philosopher asserting that iron boats are impossible, because how could they float?

Do you see a reason why a mechanical arrangement of atoms could feel pain or see color? Because they do.

Scribbles of paper serving as a part of a conscious system is indeed an implausible scenario. You'd need a huge number of pieces of paper, interacting in very interesting ways. It's an example specifically designed to be implausible, and so is a very poor guide for intuition.

> Do you see a reason why a mechanical arrangement of atoms could feel pain or see color? Because they do.

That's the hard problem.

Why wouldn't it? Supposing that it doesn't presupposes that somewhere along the gradual chain of replacing a human brain with this paper processing system those properties are lost either gradually or abruptly.

Assuming it's the functions the brain performs that are conscious. But it's part of the hard problem. Why and how is anything conscious in the physical world? What does that mean for other physical arrangements? How would we know?

> Assuming it's the functions the brain performs that are conscious.

As far as I know, if you turn off someone's brain functions you also turn off their consciousness. Moreover, consciousness can be manipulated by manipulating the brain. It stands to reason that consciousness is a subset of brain function (and body, sure). But I concede that this is not a rigorous proof of anything. Perhaps the consciousness treats the brain like a comfortable chair, and when it is destroyed gets huffy and leaves to go somewhere else. Seems unlikely though.

> But it's part of the hard problem. Why and how is anything conscious in the physical world? What does that mean for other physical arrangements? How would we know?

Hard to state, perhaps. Hard to answer? I suppose we'll find out once stated whether the question is actually hard to answer.

Relevant comment that argues in this direction: https://news.ycombinator.com/reply?id=23221295

The original claim was lots-and-lots-of-people calculating on many pieces of paper. But it's just an example of this endless stream of sort of ridiculous bait-and-switch arguments. First someone, says "so it's mechanical, so a big, fast, accurate speak-and-spell could simulate it" and you say "well, OK, it would have to be huge, incredibly fast and accurate and so-forth" and then the person "well, I doubt consciousness is just a speak-and-spell.

If the claim is that certain functions are what "generates" consciousness, then it's fair to ask whether any sort of functional arrangement will do, including writing stuff out on paper. I don't think it matters whether a billion Chinese are busy writing out 1s and 0s, or a computer is moving electrons around. None of that is conscious in my view, or at least I see no reason it would be.

I would sort of agree. Just avoiding first positing "well, even something silly would do" and then saying "well, that proves it's impossible 'cause it's silly". Then the argument is clearly unfair.

Your billion-person computer certainly would not appear to be conscious in real time - maybe that's why you just can't get your mind around the picture.

Can you imagine a billion people, none of whom who have never even heard of chess, writing out 1s and 0s, and thereby beating a grand master at the game? (in an appropriately slowed-down game, of course.)

Yes.

There’s a short story from 1981 by Vernor Vinge titled “True Names”. Recommended reading.

What makes electrochemical and chemical reactions in a brain so special that you see a reason for them to create consciousness? Or is it a lack of complete understanding of those processes that makes you think so?

Well, to a panpsychist, there's no reason why "conciousness" would need to stop at a human scale. The universe as a whole might be concious, and part of that concious experience might be isomorphic to the subjective experience of the brain that's being replicated on those pieces of paper. Though I can see how people might object that this is merely conflating "conciousness" with "computation", that's really inherent in what the thought experiment is doing.