News for hackers. Stuff that matters.
Re:Deep Learning in the Real World: Dealing with Non-Differe... by dimatura on Monday November 04, 2019 @11:23AM (#21443072)
In my opinion, the point about automatic differentiation isn't quite correct. Automatic differentiation will help you compute gradients for complicated functions involving multiple operations, as long as each of these operations are themselves differentiable (recursively; at some point, there's basic 'building blocks' with hard-coded gradients). But not all operations are inherently differentiable (in fact, most probably aren't). Now, there are other ideas such as neural Turing machines and differentiable programming, but these are not really what you'd refer to when talking about autodifferentiation.

There are no comments on this comment yet.