We can objectively observe that there is "stuff" happening in your brain, and we can objectively observe that GTP does a lot of "stuff" during its forward pass and then stops and does nothing--nothing is happening outside those forward passes.

If we hook GTP into an infinite loop and stuff resembling intelligence comes out, then we're really going to have to start questioning what consciousness is. It's coming.