Re:Differentiable Programming Manifesto
by noblethrasher on Wednesday November 13, 2019 @10:34AM
(#21525236)
He also gave nice presentation on AD: https://www.youtube.com/watch?v=ne99laPUxN4 (surprised that you didn't already mention it, considering the venue ;-))
Re:Differentiable Programming Manifesto
by kragen on Wednesday November 13, 2019 @11:50AM
(#21526197)
The manifesto links one of his papers, "The simple essence", which I've started but haven't finished.
Re:Differentiable Programming Manifesto
by seanmcdirmid on Wednesday November 13, 2019 @12:42PM
(#21527014)
He has been working on that for a long time. Though much of his earlier work is symbolic differentiation, I’m not very familiar with the more recent AD stuff.
Re:Differentiable Programming Manifesto
by thechao on Thursday November 14, 2019 @08:20AM
(#21535129)
Let me mostly summarize Conal’s “simple” reverse AD: he drops the tape into the stack. This has two impacts: the stack is “about twice as long”; and, you “may (will?) lose tail recursion”.
It is still a really good paper—and no doubt a lot of the subtlety was list to me.
He is now very active in the differentiable programming community, but he got started on the problem in the graphics field.