News for hackers. Stuff that matters.
Re:Automatic Differentiation with Julia by aaaaaaaaaab on Tuesday October 23, 2018 @11:15AM (#18284052)
To calculate the gradient of a function R^n -> R^m, forward-mode AD is preferable if n << m, while reverse-mode is faster for n >> m.

From this it should be clear why machine learning uses reverse-mode.

On the other hand, forward-mode is better for e.g. calculating the tangent of a high-dimensional curve (i.e. R -> R^n).

There are no comments on this comment yet.