For the avoidance of doubt, the intention of the thought experiment is that it does indeed make accurate and precise predictions, so that if you give the real person and the simulated person the same experiences, they respond in the same way.
Of course in practice, even with all the technological advances that would be required to make the thought experiment a reality, you'd never be able to make the experiences, or the starting states, exactly identical. So in cases where how you act is exquisitely sensitive to the details of your starting state or the experiences you have, simulation and reality might diverge. My guess is that many things aren't so delicate, and I personally would be pretty impressed by a me-simulation that consistently said and did things that are the sort of things I would be likely to say and do.
I'm assuming we're ultimately talking about the capabilities of the human mind - comprehensively, it's capabilities are significant. But then, we also know it to be incredibly flawed, which should be kept in mind during such considerations.
> For the avoidance of doubt, the intention of the thought experiment is that it does indeed make accurate and precise predictions, so that if you give the real person and the simulated person the same experiences, they respond in the same way.
So something kinda like this (way more complex likely)?:
"select distinct [phenomenon] from [human_mind_capabilities] where [type] in ('experience','response')"
I wonder how many rows that would return. I also wonder how much diversity would be in the resultset. Either way, I think one would need a pretty powerful thought experiment for even remotely comprehensive coverage.
> So in cases where how you act is exquisitely sensitive to the details of your starting state or the experiences you have, simulation and reality might diverge.
One example (singular, at least kinda):
https://en.wikipedia.org/wiki/Jeffrey_Dahmer
Another (when networked):
https://en.wikipedia.org/wiki/Casualties_of_the_Iraq_War
> My guess is that many things aren't so delicate, and I personally would be pretty impressed by a me-simulation that consistently said and did things that are the sort of things I would be likely to say and do.
I too have an intuition that there is a very rich vein of ore at some level, and that it may be accessible, were we to look for it.
> Question 1: Do you agree that something along these lines is possible in principle?
Seems reasonable.
> Question 2: If it happened, would you think there is still a "hard problem" left unsolved?
Yes.
> Question 3: If you think there would still be a "hard problem" left unsolved, is that because you think someone in this scenario could imagine all the machinery revealed by the simulator operating perfectly without any actual qualia?
If I see an amazing demo of something, and then they try to sell me something that is capable of only a small subset of what was just shown in the demo, I am going to balk. I suspect if the topic was most anything other than consciousness (which is one of those topics that seems to cause the mind to behave anomalously for some reason), I think most people would agree.
EDIT: thinking about it more....if you could start to make amazingly accurate and precise predictions about how the mind is going to react under a wide variety of scenarios, I would be much more impressed...although, my intuition is that this is far less complex than it may seem for normal behaviors (fine-grained precision is where I'd have to surrender my skepticism).