That depends on how you define AGI. You'd probably want to give it a few more senses, so it can have unwritten qualia, or at least so you can show it pictures.

More importantly, current ML models can't decide to think about things, because they always spend the same processing time on everything. GPT is recurrent and you do sort of feed its output through it, but there isn't a global context it uses for the whole document.