Let's consider a simple image classifier: 64x64 grayscale input, 10x1 output that detects 10 classes of images. The model: y=tanh(Ax+b). You'd probably say that there is no way this will work because this model is too simple. But can you explain why this won't work? Can you tell what the maximum accuracy this model can reach? What kind of datasets this model would work better on?

[Edit] Whether this model works is question. Sure, it can't recognize dogs or cats, but what if the dataset is 0-9 digits? Now it suddenly works, right? And works really well. But what's changed? Can we describe in mathematical terms what makes the 0-9 dataset so special? It'll probably work with A-J letters too, but what about hieroglyphs?

The math I'm looking for would tell that E(T) is the error and on such model and such dataset, E(T)=exp(-T^2)+O(exp(-T^3)), according to such and such theorem; and according to another theorem, if the dataset is isomorphic to that manifold, E(T) can be improved to O(exp(-3T^2)).

I think you're asking for the holy grail here. Everyone would love to have such a thing but nobody thinks it's likely to be possible.

So they settle for much smaller targets. Either of understanding how much simpler systems work. Or of trying to understand a little bit the effect of tweaking something in some more complicated model.

Perhaps you should think of these two approaches as analogous to doing simple chemistry (what shape is a sugar molecule? A DNA molecule?) vs trying out drugs (if you eat the bark of this tree, you don't get malaria! Let's refine that stuff). Both can be useful, but they are very far from a unified theory of how your body works.

ML is more like alchemy, I'd say: mixing components using intuition and experience, but without understanding what these components really are and why they work. In this analogy, AI is the recipe to make gold and the ML alchemists haven't invented nuclear physics yet.

But now we know there was physics at the bottom of alchemy. Whereas demonology at best leads you to psychiatry, and we still don't have simple models of what works there. Nor much hope of finding them. Thinking is a messy business.

My guess is that what I'm asking for isn't that complex and could be done by a few serious mathematicians in a few years. The dynamics of tanh(Ax+b) is hardly more complex than Naiver-Stokes equation or the modern topology theory.

tanh(Ax+b) is simple, but the dataset it's supposed to work on is not easily summarised. I think that's the huge difference. The guys doing "sugar molecule" studies make progress by taking much simpler datasets, like random points.

Naiver-Stokes is much simpler because it operates by itself. Of course turbulence is hard but even there we usually care about its coarse features, we'd be content to throw away almost all the information provided the calculation of the wing's lift works out OK.