Re:Backpropagation is a leaky abstraction
by joe_the_user on Monday December 19, 2016 @10:23PM
(#13217198)
Perhaps the distinction would be between things that are "fundamentally a heuristic with some math behind it" versus things are "mostly theoretically settled with some corner cases you have to worry about"
And neural networks using backpropagation to minimize an error function are definitely the former while the "go to" techniques of mathematics are generally the latter.
I think this is generally acknowledged, in fact.
Sooo it seems logical that students of these modern machine learning techniques should be impelled to "get their hands dirty" with the numericals processes involved since this will add to the data they use to drive the intuitions which allow them to avoid the multiple pitfalls of neural networks.
And neural networks using backpropagation to minimize an error function are definitely the former while the "go to" techniques of mathematics are generally the latter.
I think this is generally acknowledged, in fact.
Sooo it seems logical that students of these modern machine learning techniques should be impelled to "get their hands dirty" with the numericals processes involved since this will add to the data they use to drive the intuitions which allow them to avoid the multiple pitfalls of neural networks.