"Assume it's possible to simulate a conscious brain with a deterministic program."

I think this is touching on the 'philosophical zombie' arguments.

If something is crated like a robot/ai, that is completely indistinguishable from a human, is that thing even possible without having some inner subjective experience? It seems like people fall into two camps, if it is a machine, then of course it is a zombie, no inner life. The second camp that says the machine is conscious. The problem is from outside, you can't really prove it either way because the premise is that it is indistinguishable.

So, I think in your example, it is interesting extrapolation, the 'invention' is indistinguishable and deterministic, now what if we split it up, compress it, etc.. when would that inner experience go? or would it end at some point?

Of course, if a consciousness could be played back and forth like this, then that would be argument for GPT to have some consciousness .