Re:Differentiable Programming – A Simple Introduction
by SleekEagle on Wednesday April 13, 2022 @11:39AM
(#31015995)
Yes exactly! This is huge. Hessian optimization is really easy with JAX, haven't tried it in Julia though
[ Parent
| on HN ]
Re:Differentiable Programming – A Simple Introduction
by ChrisRackauckas on Wednesday April 13, 2022 @01:19PM
(#31017358)
Here's Hessian-Free Newton-Krylov on neural ODEs with Julia: https://diffeqflux.sciml.ai/dev/examples/second_order_adjoin... . It's just standard tutorial stuff at this point.
Re:Differentiable Programming – A Simple Introduction
by PartiallyTyped on Wednesday April 13, 2022 @01:15PM
(#31017305)
And very fast given that you compile the procedure! I am considering writing an article on this and posting it here because I have seen enormous improvements over non jitted code, and that excluded jax.vmap.
Re:Differentiable Programming – A Simple Introduction
by SleekEagle on Wednesday April 13, 2022 @01:53PM
(#31017866)
There's a comparison of JAX with PyTorch for Hessian calculation here!
https://www.assemblyai.com/blog/why-you-should-or-shouldnt-b...
Would definitely be interested in an article like that if you decide to write it