Foundations Built for a General Theory of Neural Networks

Re:Foundations Built for a General Theory of Neural Networks by nora4 on Sunday February 03, 2019 @06:17PM (#19072858)

Deep Learning is currently an empirical science guided by intuition of practitioners. A main principle in experimental sciences is that a theory without predictive power is not considered a full-fledged theory. As such, unless they are interesting predictions coming from their theory (rather than only barely justifying existing empirically observed phenomena), this is just speculative theory that I would not use the phrase "Foundations Built" for.

As an example of this general litmus test for a theory see e.g. Eddington's confirmation of GR: https://en.wikipedia.org/wiki/Tests_of_general_relativity#De... . If there are hitherto unknown phenomena in DL predicted by this theory then I'd stand corrected and concede that there may be something to these theories.

[ Parent | on HN ]

Re:Foundations Built for a General Theory of Neural Networks by fizx on Sunday February 03, 2019 @06:33PM (#19072905)

Read this as "some foundations built". Even that is much-needed progress.

[ Parent | on HN ]

Re:Foundations Built for a General Theory of Neural Networks by hyperpallium on Sunday February 03, 2019 @09:40PM (#19073570)

Yeah, it's not foundations like elementary or axiomatic.

An actual theory of ANN and then of NNN will be of more momentous import than of any previous phenomena, will usher in a fundamentally transformative age where we understand ourselves, and will surely require a fundamental breakthrough in pure mathematics... probably a great many.

[ Parent | on HN ]

Re:Foundations Built for a General Theory of Neural Networks by matt4077 on Sunday February 03, 2019 @08:51PM (#19073410)

The article seems pretty upfront about the fact that we are nowhere near a solid theory. One of many quotes to that effect: “ […] beginning to build the rudiments of a theory of neural networks.”

The main point seems to be why such a theory is highly desirable. To that end, it does a far better job than the somehat tired Feynman aphorism about good theories “giving us more than we give them”. Namely that the current state of the art of “intuition” and vast trial-and-error runs being somewhat embarrassing for a field so closely related to math and statistics.

[ Parent | on HN ]