In David Deutsch's book The Beginning of Infinity, he argues, that the purpose of science is not to merely predict or to model the world, but to explain it. And he gives in my opinion convincing arguments, why a better explanation is needed to advance science. Therefore, he deduces that pure empiricism has failed. Because no matter how much you have gathered, an explanation does not follow from it. The explanation is always an insight the scientist has to contribute. Then it is open to scrutiny by criticism or experiments.
I guess he would argue the same is true for mathematicians. Merely constructing proofs to arrive at theorem, is of no intrinsic value, if it doesn't spark an idea of a more elegant formalism that corresponds to a simpler and even more far reaching explanation.
I cannot recommend this book highly enough to anyone who is intellectually curious, not only about epistemology.
That's not to say there isn't value in building mathematical insight for humans, just that there is a goldmine and a lot of people know what to do with gold, even if they don't understand the pickaxe.
One bit stands out to me: to better understand, Scholz will work the problem backwards. In a lot of circumstances, that strategy is just so spot on.
To my eternal shame, philosophy makes my head hurt. Pretzels my brain, ouroboros like, in other words "head up my own butt". That said, here's my cave man cold take on two aspects of this topic.
#1 As we're all trying to paraphrase here: answers are great; understanding is even better. I like the word "legibility" (h/t James Scott). The current models are not very legible.
Which directly leads to:
#2 Understanding enables (humans to make) predictions.
If LLMs are stochastic parrots, then pi.dev (GPT + reasoning + ralph loop) is a drunken sailors walk (thru a solution space). A kind of messy heuristic for optimization. With some kind of gate for declaring "done!" Pretty useful.
However, I don't expect Pi.dev to make predictions. That'd require something else. Maybe that something is being worked on; am noob, so I wouldn't know. (I'm still unclear about whatever Jeff Hawkins is doing, in practical terms.)
Are there Scholz-like tools to work backwards thru a problem? Again, I don't know.
--
Wrapping up: Automated prediction would be cool. That's very different from finding solutions to problems. The current cohort of AI can't even explain why, much less make predictions.
(Someone please correct me as needed. TIA.)
[Aside: That article has an ad for their podcast They Joy of ₍₎. Perfect. I have no idea if it lives up to its promise.]