I think it's more about the lack of data. GPT-4's training likely already used all publicly available text on Earth and some private databases too.