Theory is for us, so we can understand what the models are doing. They gave the example of Fable, which understood and explained everything perfectly right away (written by OpenAI's unreleased model in terse mathematical notiations).
This was totally intuitive for the AI (Fable), but it took this person two weeks just to get a high-level grasp of it. And this person is one of the few people on earth who can (he is an expert in this niche of mathematics).
https://web.maths.unsw.edu.au/~davidharvey/research/subnlogn...
Controversial statement: I think we've always done math by generalising from concrete experience. So concepts like numbers, spaces, shapes, distances, and curvature are based on physical experiences we use as anchors before expanding them far beyond physicality and looking for patterns in the abstractions.
If AI is generalising from patterns in latent space, that's a different kind of abstraction, and one that's much more alien to us, because it's going to be much, much bigger - too big to fit inside even the smartest human mind.
So AI theories and abstractions may be valid in true/false terms, but we will never be able to comprehend them in their raw form - just as we can't experience latent space directly, and will only experience tiny slices of it as outputs to our queries.
Ergo, there are likely numerous deeper insights that will emerge when smarter models have the chance to go over brute forced, millions of line Lean proofs. Likely things humans have not conceived, but I believe will truly advance understanding.
Compare Elliot Glazer a month-ish ago:
> I occasionally talk to experts on all the MP's. Their rough summaries on what is needed to solve them usually go:
> Hodge: not a clue
> BSD: not a clue
> RH: not a clue
> P vs NP: not a clue
> Yang-Mills: "Well..."
https://news.ycombinator.com/item?id=50252673
So if we're in "not a clue" territory, maybe that's that.
I fear that it gets muddled again with e.g. the concurrent human announcement and that in retrospect RH turns out to be easier than expected, like many other problems before.
Math is hard, but how hard math is is also hard...