To quote/paraphrase Leonard susskind: people have a habit of philosophizing rather than sciencifizing when it comes to consciousness. It's scientific question - or at least it is at the moment. Some people want to enshrine human intelligence (i.e. free will and quantum mechanics), which I think is naive without some kind of guiding principle (Fermi's "for a successful theory you need either a strong physical idea or a rigorous mathematical basis; you have neither" to a young Dyson)

I remember being taught about this in a philosophy class, but I really can't help but not care - if it is technically possible I guess there will be a debate surrounding machine consciousness similar to transgender politics today (Uncomfortable to some but fine to - including me for the record - others).

"if the simulation is accurate, then the person you're simulating is conscious"

Not necessarily. You can't treat a human as a black box, and then use inputs and outputs to definitively draw conclusions about what's going on inside the black box.

You could potentially have two black boxes that give identical outputs for the same inputs, yet have completely different mechanisms inside for arriving at those outputs. For example, say you have someone who's very good at arithmetic able to multiply two numbers. You put him inside one black box and a pocket calculator inside another. You give each box two numbers and they both output the product. Both black boxes will give you identical outputs for any given input. You know the box with the person inside is conscious, but this is not enough information to conclude the other black box is conscious.

"Consciousness" is not a descriptor of the outputs of a system. Consciousness is a descriptor of how it processes the inputs to arrive at the outputs.

There is a thought experiment in psychology related to the subject called the p-zombie https://en.wikipedia.org/wiki/Philosophical_zombie. "Such a zombie would be indistinguishable from a normal human being but lack conscious experience, qualia, or sentience. For example, if a philosophical zombie were poked with a sharp object it would not inwardly feel any pain, yet it would outwardly behave exactly as if it did feel pain." You're basically arguing that it's impossible for a p-zombie to exist. That is the view some people hold, but arguing for that case is a lot more complex than simply saying its a tautology

I didn't mention the inputs or outputs, I mentioned a perfect simulation in all biological details. That's the opposite of a black box, you'd be able to freely pause and inspect its behavior. So, I don't see why what you posted is relevant.

As for P-Zombies, there would again be no difference between this brain, and a stranger you pass by on the street. Either of them could be a p-zombie, but we give other humans the benefit of the doubt; the same biology should lead to the same internal experience. Personally, I'm completely unconvinced by the p-zombie concept in general, but that's not relevant either.

Indeed, it should be possible to simulate all the "required" physics on a computer. (Not that it's necessary.)

My totally unsubstantiated and made up take on this is that perhaps consciousness happens in the quantum side of things that we can't either measure or really simulate because we can never really observe it, e.g. we can simulate quantum behavior quite well, but only based on the observed behavior, the particle side of the particle wave duality, we don't really know for sure what happens before the wave function collapses, we just know it's a probability space (which we can predict for repeated experiments very accurately, but we don't really know how to predict a single particle, we don't even know how to measure all of its properties at the same time let alone predict it). If consciousness happens in that no locality / no realism area of the unobserved quantum state, then perhaps this is why we can't simulate it with existing technology. I have no basis to this thought other than a hunch. It also somewhat helps explain determinism vs free will (although superdeterminism is also an explanation, one that I don't really like)

"... we don't even know how to measure all of its properties at the same time ..."

I assume you're referring to Heisenberg's Uncertainty Principle [1]. If you are, then this shows a fundamental misunderstanding of it. The uncertainty principle describes a fundamental property of quantum systems. It has nothing to do with our measurement technology.

[1]: https://en.wikipedia.org/wiki/Uncertainty_principle

You’re throwing disconnected buzzwords around.

The brain is too hot to rely on large-scale quantum entanglement. Any quantum effects that are relevant to the functioning of the brain are likely to be very small-scale and have some boring, easily-simulated effect like making ion pumps more efficient.

The logic appears to be simple: consciousness is mysterious, but so is quantum world, therefore they must intersect somehow.

To quote John Searle:

"Nobody supposes that the computational model of rainstorms in London will leave us all wet. But they make the mistake of supposing that the computational model of consciousness is somehow conscious. It is the same mistake in both cases."

According to Wikipedia, John Searle is known for sexual assault, sexual harassment, and the "Chinese room" argument. I'm not sure with which of these he's done the most harm, but I suspect he's done the most harm with his "Chinese room" argument, primarily through wasting people's time. See https://en.wikipedia.org/wiki/Talk:Chinese_room for some discussion, if you must, but I'd rather recommend people do something more useful instead.

Could you point out exactly where I can see a sound refutation to the 'Chinese room'? I followed the wikipedia link but did not see any serious attempt to refute it - just one guy echoing what you've said.

You sound a bit like many of the people who completely miss the point of the argument and insult it (or its author).

The Chinese room argument says that syntax cannot capture semantics since a man blindly executing rules to process Chinese symbols would not understand Chinese. But the man isn't the the system that is purported to understand Chinese. That's like saying the CPU in a computer doesn't have memory so your computer doesn't have memory. But of course that's wrong, other components of the system provide the function of memory. Using properties of the CPU to determine properties of the system as a whole is a mistake.

You might recognize this as the system's reply for which Searle has a response. But his response is insufficient to save the argument as it merely equivocates on what "the system" is. The system is the set of rules and the database of facts, some input/output mechanism, and some central processor to execute the rules against the symbols. The system's reply says that the man blindly executing instructions is not identical to the system that understands Chinese, namely the entire room. Thus properties of the man have no bearing on the properties of the system; that the man does not understand Chinese has no bearing on whether the system as a whole understands Chinese.

But the man is still not "the system" when he memorizes the instructions and leaves the room. The system is some subset of the man, specifically only those brain networks required to carry out the processing of the symbols. Importantly, these brain networks are different than what we take the man to be. It is true that the man doesn't speak Chinese, but since the system processing symbols is still not identical to the man, properties of the man do not bear on properties of the system. So still, the fact that the man blindly executing instructions doesn't speak Chinese does not entail that the system doesn't speak Chinese.

To see an analogy here is to see that there is something fundamentally biological about consciousness. But this is very unlikely to be true. For example, we can see how there is something fundamentally watery about rain. And so a rainstorm without water will not leave us wet, thus is not a genuine rainstorm. But what is the analogous biological substrate necessary for consciousness? Searle has never clarified.

But we know a lot about the biology of the substrate of consciousness, and none of those properties seem relevant to consciousness. There doesn't appear to be any fundamental property of neurons that is relevant to conscious experience. The only thing of relevance is its organization and thus its causal or computational structure. But these properties can be reproduced arbitrarily well in a general computer.

That seems rather a duff argument. Human weather forecasters predicting rain don't make us wet either. It doesn't show they are not conscious.

This is not about how human forecasters or computer weather models are conscious.

The idea is if the cloud formations and rain generated by the model are indistinguishable from what you are seeing from your window. It doesn't mean that the model is actually producing rain. It may look the same on camera, but if you go outside, it is very different: only the real rain will make you wet.

In the same way, if a computer behaves like a conscious entity. For example a chatbot acting like a human, as in a Turing test. It doesn't mean that the computer is conscious.

Note that the computer may be conscious, but it is not a requirement in order to run a model. In the same way that a weather model could use actual water, but doesn't need to.

Do you think all there is to consciousness is biology? That's where the "accuracy" comes from, so if the respondent disagrees with that as a fundament, then it's a reasonable answer.

This is what bugs me about the Turing test; it doesn't answer the important, hard question of AI: is something conscious, and to what extent does it have life.

The point of the Turing test is if you can't tell the difference then it doesn't matter.

As of today you can't prove you're not in a simulation nor can you prove that every person you think is intelligent is not really an AI in that simulation.

So similarly, if an AI exists and you can't tell it's an AI no matter how hard you try then there is no reasonable difference. The are effectively the same.

The only point to Turing's test is to be double blind. You can't know before you start that the other side looks like a computer not a human or that they're voice is off or that if you cut them open they don't bleed blood etc. You have to make the test double blind and to do that a chat format is easiest.

My interetation: This is the point made by Turing. One can only evaluate input/output. Analysis of methods/structure used by AI can not tell you if that intelligence is real or not, or is there a difference at all.