News for hackers. Stuff that matters.
Re:Composability in Julia: Implementing Deep Equilibrium Mod... by ssivark on Thursday October 21, 2021 @01:35PM (#28948018)
The original Neural ODEs paper is quite readable, and by now there are loads of blog posts and even a few talks on the subject.

The basic idea is inspired by the “adjoint method” for ODE solving (so you don’t have to hold in memory all the intermediate layer outputs — which is otherwise necessary to compute the backpropagated gradient signal).

Re:Composability in Julia: Implementing Deep Equilibrium Mod... by ChrisRackauckas on Thursday October 21, 2021 @02:37PM (#28949077)
Yeah, though with the method described in that paper you do have to be very careful since it has exponential error growth with the Lipchitz constants of the ODE. See https://aip.scitation.org/doi/10.1063/5.0060697 for details. But that is generally the case in numerical analysis: there's always a simple way to do things, and then there's the way that prevents error growths. Both have different pros and cons.