Re:Differentiation for Hackers
by nthompson on Saturday June 22, 2019 @01:57PM
(#20250844)
Automatic differentiation only works on functions which are locally complex analytic. It fails on things that fall outside that model, but are still differentiable. Daubechies wavelets are a good example of this.
[ Parent
| on HN ]
Re:Differentiation for Hackers
by ChrisRackauckas on Saturday June 22, 2019 @06:31PM
(#20252059)
Julia's systems can make use of ChainRules.jl which doesn't assume locally complex analytic, and instead uses its Wirtinger derivatives (df/dz and df/dzbar), and if complex analytic just uses the well-known simplification df/dzbar=0. This allows for non-analytic functions to be used, and you just need to supply all primitives in Wirtinger form. Julia has enough non-ML people using this stuff that complex numbers actually get used here :).