Just musing, so don’t take this as a direct response to your comments…

I agree with you but for rhetorical necessities it would be great if this argument could be made without the direct comparison with human cognition if only because there’s a popular grey-faith argument that will dismiss this offhand and for any number of deeply held philosophical beliefs.

The idea that emergent machine cognition mirrors emergent biological cognition is bordering on behavioralism but on the Wittgenstein side of things. The forest for the trees.

Perhaps it’s not as pithy as you’ve laid out, which is not an insult, just an observation that side-stepping anthropomorphism isn’t going to be as straightforward.

What if consciousness is an emergent illusion? “You” are merely a passenger, observing your physical self’s actions and assuming ownership of them. What if consciousness is merely an effect, not a cause?

I hope you haven't read the novel blindsight and I get to be the one to tell you about it. They might as well have given that book your post as a title.

>“You” are merely a passenger, observing your physical self’s actions and assuming ownership of them

Oh boy. Now here's the rub. We know this is true at least sometimes. When you make a decision and explain it, it is often(always?) just a post-hoc rationalization. You don't actually know why you make a lot of the decision you do even if you dearly believe so.

This is very clear in split-brain patients where one side will make the decision and the other side will come up with an excellently reasoned explanation of the decision.

What's even weirder is that when the hemispheres are split apart, the two sides seem to both be able to understand and carry out instructions (and can even compete with each other to do them, e.g., the left hand and right hand may try to push each other out of the way), which suggests that there may be two wholly separate consciousnesses in each half of the brain.