News for hackers. Stuff that matters.
Re:Deep Learning Algorithms: The Complete Guide by rckoepke on Wednesday March 04, 2020 @03:04PM (#22487466)
Mainly I take issue with the title "...: The Complete Guide". More accurately, its a very abbreviated overview. The table of contents has a fairly reasonable list of topics, but for a complete guide I'd want each section there to be at least an entire chapter of material. Instead they are 4-6 sentences per section.

Personally, I think that any comprehensive guide/textbook for machine learning, especially deep learning, should contain a chapter on the mathematics of control systems theory. The feedback/back-propagation is very similar to what EE's and ChemE's do with PID loops or Op-Amp tuning, and I feel that so much is lost academically when that topic is glossed over.

Re:Deep Learning Algorithms: The Complete Guide by throwqwerty on Wednesday March 04, 2020 @05:44PM (#22488975)
>The feedback/back-propagation is very similar

i really wish people wouldn't say these things just to sound smart. it confuses people that don't know better. feedback loops have nothing to do with backprop, which is just a way to factorize the jacobian into matrix vector products instead of matrix matrix products. do you have some proof (and it would require a proof) that PID controllers actually compute the gradient of a function with respect to n-inputs? because that's what backprop does.

Re:Deep Learning Algorithms: The Complete Guide by rckoepke on Thursday March 05, 2020 @07:45AM (#22493053)
Thank you, non-sarcastically. Would edit if I could.
Re:Deep Learning Algorithms: The Complete Guide by joe_the_user on Wednesday March 04, 2020 @04:09PM (#22488150)
I'm not even sure Deep Learning is at the level where a title like "The Complete Guide" could make sense.

Deep Learning involves harnessing advanced, highly complex and rather ad-hoc algorithm to engage in systematic-but-heuristic prediction.

A complete guide would include all natural prerequisites, the common approaches, the best practices and the areas of application. But all of these are in flux as the field races ahead. Moreover, the field still needs "artists", practitioners who can figure out the black-art of training networks. So whatever it's virtues, the field today seems incomplete to me.

Re:Deep Learning Algorithms: The Complete Guide by jjoonathan on Wednesday March 04, 2020 @03:46PM (#22487888)
Yeah, pretty much everyone can benefit from a high-level grasp of phase margin and gain margin. Even in contexts where exact calculations are impossible, it's important to remember that cranking the gain too high turns your amplifier into an oscillator and that lengthening the feedback cycle reduces the gain ceiling above which this happens.

Typically, I find that people have an intuitive understanding of the possibility of overshoot but not of oscillation and almost never of the connection between feedback lag and overshoot/oscillation.

Re:Deep Learning Algorithms: The Complete Guide by SubiculumCode on Wednesday March 04, 2020 @03:42PM (#22487837)
I liked the birds eye view of this article, but that title
Re:Deep Learning Algorithms: The Complete Guide by AgentMatt on Wednesday March 04, 2020 @04:14PM (#22488209)
Do you have a recommendation for a book explaining the relevant control systems concepts?