https://web.maths.unsw.edu.au/~davidharvey/research/subnlogn...
Controversial statement: I think we've always done math by generalising from concrete experience. So concepts like numbers, spaces, shapes, distances, and curvature are based on physical experiences we use as anchors before expanding them far beyond physicality and looking for patterns in the abstractions.
If AI is generalising from patterns in latent space, that's a different kind of abstraction, and one that's much more alien to us, because it's going to be much, much bigger - too big to fit inside even the smartest human mind.
So AI theories and abstractions may be valid in true/false terms, but we will never be able to comprehend them in their raw form - just as we can't experience latent space directly, and will only experience tiny slices of it as outputs to our queries.
Ergo, there are likely numerous deeper insights that will emerge when smarter models have the chance to go over brute forced, millions of line Lean proofs. Likely things humans have not conceived, but I believe will truly advance understanding.
Theory is for us, so we can understand what the models are doing. They gave the example of Fable, which understood and explained everything perfectly right away (written by OpenAI's unreleased model in terse mathematical notiations).
This was totally intuitive for the AI (Fable), but it took this person two weeks just to get a high-level grasp of it. And this person is one of the few people on earth who can (he is an expert in this niche of mathematics).