You are right, but the problem of a possible analogous "Machine Learning Theory" is that machine learning problems are highly nonlinear, and control theory has only been "solved" for linear problems. It we could solve nonlinear control that could immediately be applied to machine learning.
“Linear systems are important because we can solve them.” — Richard Feynman
Re:Tune Your PID Controller
by dreamcompiler on Friday May 28, 2021 @09:16AM
(#27314972)
Fair point. But of course the reasons for the nonlinearities are different. ML is not inherently nonlinear; ML is only nonlinear because linear neural nets are not very useful.
On the other hand linear control theory is quite useful for many real-world problems.
If we had an analogous "Machine Learning Theory", we'd be able to predict from first principles how many neurons we needed, how many hidden layers we needed, and what architecture (i.e. recursive, convolutional, etc.) we needed to solve a given problem. We don't have that now, but I hope we do some day. The good news is we do have such a theory for control problems (at least the linear ones).
[0] https://en.m.wikipedia.org/wiki/Control_theory