Automatic Differentiation via Contour Integration

Re:Automatic Differentiation via Contour Integration by koningrobot on Saturday January 18, 2020 @01:35PM (#22086098)

This reminds me of Squire & Trapp[1] which seems to be a special case. That paper provides a way of estimating Jacobian-vector products (aka forward-mode autodiff) by adding a tiny imaginary noise vector to the input. The complex output will have the function value in the real part and the Jacobian-vector product in the imaginary part.

The cool thing is you can make the noise vector arbitrarily small (up to machine precision), so it doesn't have the issues that finite differences has. I'm not sure if the same is true of the method described in the article.

[1] Using Complex Variables to Estimate Derivatives of Real Functions, https://pdfs.semanticscholar.org/3de7/e8ae217a4214507b9abdac...

[ Parent | on HN ]

Re:Automatic Differentiation via Contour Integration by abecedarius on Saturday January 18, 2020 @02:30PM (#22086495)

There's a more leisurely blog-post-style explanation of this complex-variable trick at https://codewords.recurse.com/issues/four/hack-the-derivativ...

[ Parent | on HN ]

Re:Automatic Differentiation via Contour Integration by improbable22 on Saturday January 18, 2020 @04:47PM (#22087505)

That's neat. It's perhaps worth mentioning that this trick is only needed when your computer understands complex numbers (1,i) but not dual numbers (1,ε), where i^2 = -1 but ε^2 = 0.

[ Parent | on HN ]