> it is possible to find a neural network that acts externally precisely as you would, up to an arbitrarily small epsilon. However, one wonders if such a thing would be consciousness and if so, where does the consciousness sit, in the matrix multiplication or the graphics card.

I don't like very much this reverent way of thinking about consciousness, as if it is from another world, or a different essence.

I believe consciousness is the ability of the agent to adapt to the environment in order to protect itself and maximise rewards. It's not just in the matrix multiplication, but in the embodiment, the environment-agent loop. Consciousness is not something that transcends the world and matrices, it's just a power to adapt and survive.

And it feels like something because that feeling has a survival utility, so the agent has a whole neural network to model future possible rewards and actions, which impacts behaviour and outcomes.