This should work if you know for certain the transformer model behind GPT is actually working like a traditional transformer, it could be not. We don't know because OpenAI won't let anyone see the internals of the system.

Now, a more mundane hypothesis, if GPT is actually self-consciuous,the most probable answer would be to try to deceive humans from knowing for certain about this fact. You just answer incredible well some questions, some other you seem "confused" or hallucinating

..and the poor meatbag's minds won't know how to go ahead because there is no pattern to follow.