But add an AI superresolution model after that and you're off to the races!

The problem with superresolution is that by design, they focus on details and textures for upscaling, and avoid adding in any large-scale structure. So if you train it separately (ie there is no backpropagation from the final highres 224px image through the superresolution model all the way back through iGPT), you cripple the original model.

There is only so much about objects and the world you can learn from 64px thumbnails, and they mention that this is probably a reason iGPT wins on the tiny images like CIFAR but loses to the semi-supervised CNNs on ImageNet: because the CNNs are compute-efficient enough that they can train at the standard 224px and see all of the details and structure that disappears at iGPT's 64px, and learn end-to-end.