> some people will say it's just regurgitating whatever words and images and associations it has seen in training

That's why I specifically mentioned ARC: Its test test is novel and fully private (even to us, humans), so the model will need abstraction capabilities in order to solve it.