Re:Swift: Google’s Bet on Differentiable Programming
by pabl0rg on Thursday April 09, 2020 @07:39AM
(#22821570)
Is it very different from probabilistic programming (a term that is both older and easier to understand)?
Erik Meijer gave two great talks on the concept.
Re:Swift: Google’s Bet on Differentiable Programming
by zeroxfe on Thursday April 09, 2020 @08:40AM
(#22821947)
Yes, these are two very different things.
Differential programming is about building software that is differentiable end-to-end, so that optimal solutions can be calculated with gradient descent.
Probabilistic programming (which is a bit more vague) is about specifying probabilistic models in an elegant and consistent way (which than then be used for training and inference.)
So, you can build some kinds of probabilistic programs with differential programming languages, but not vice versa.
In essence there are cases outside the well developed uses (CNN, LSTM etc.) such as Neural ODEs where you need to mix different tools (ODE solvers and neural networks) and the ability to do Differentiable Programming is helpful otherwise it is harder the get gradients.
The way I can see it being useful is that it helps speed up development work so we can explore more architectures, again Neural ODEs being a great example.