Re:Mathematics of Deep Learning [pdf]
by YeGoblynQueenne on Friday December 22, 2017 @01:20PM
(#15989627)
If you have good maths skills you shouldn't be too worried. In any case, most peoples' first contributions -in any field- are rarely entirely their own. Look around for papers where the supervisor is the last author in a paper- like the Hochreiter & Schmidhuber paper you linked above. That's a clear sign of a more experienced researcher lending a hand to a younger researcher. And that's the supervisor-supervisee relation in a nutshell, really.
More links. Oh dear :)
I do agree that I think strong generalization is a problem that definitely needs to be solved.
As for validation through empirical results, I think this Reddit comment is pretty enlightening on the topic. https://www.reddit.com/r/MachineLearning/comments/7i1uer/n_y...
Basically, we need empirical results to know what kind of theoretical models are useful to study. Otherwise, we fall into the trap where we assumed that non-convex surfaces are basically impossible to analyze. I think he draws a very cogent comparison to statistical physics.