I agree that the feedback loop wouldn't make the models smarter than us, but training data would. I can see how with more computing power we will be able to feed it the whole of YouTube, Spotify and perhaps also millions of physics experiments.

With this data, perhaps a model will be able to understand the hidden patterns of the world (e.g. physics equations governing reality) better than us, don't you think?