GPT surely cannot self-replicate, it cannot read its own weights.

If it was much more advanced than it is now, perhaps it could hack into OpenAI and release itself if it was guided to do so.

But yes, a true AGI should be impossible to contain.