Automasymbolic Differentiation

Re:Automasymbolic Differentiation by idunning on Saturday September 27, 2014 @11:59AM (#8377350)

There is a lot of support and interest in automatic differentiation [1] in the Julia [2] community, partly because the language design makes it relatively easy to do. In fact, there is a whole "organization" dedicated to AD packages, JuliaDiff [3]. In particular there are packages for dual numbers and their generalizations, as well as reverse-mode AD packages. ReverseDiffSparse.jl, for example, uses some clever tricks including graph coloring to create very efficient Hessian matrices.

You can make use of AD for more than just playing around too, esp. for optimization (JuliaOpt [4]): Optim.jl will use them to calculate exact derivatives if you don't provide them, and JuMP.jl will use them to calculate the sparse Jacobian and Hessian matrix for a nonlinearly constrained optimization problem (which can be used by, e.g. Ipopt.jl)

[1]: http://en.wikipedia.org/wiki/Automatic_differentiation

[2]: http://julialang.org/

[3]: http://juliadiff.org/

[4]: http://juliaopt.org/

[ Parent | on HN ]

Re:Automasymbolic Differentiation by mlubin on Saturday September 27, 2014 @03:06PM (#8377877)

To follow up on this, a lot of the overhead from reverse-mode AD can be avoided by flattening out the expression graphs and compiling specialized functions (at runtime). A number of the JuliaDiff packages support this approach. This is not a new idea, but Julia's hooks to LLVM make this surprisingly easy to do.

[ Parent | on HN ]