Re:Stan: a probabilistic programming language
by tel on Saturday September 19, 2015 @01:09PM
(#10244848)
+1 on all of this (except I'm a fan of full MCMC, have you used ADVI?)
[ Parent
| on HN ]
Re:Stan: a probabilistic programming language
by probdist on Saturday September 19, 2015 @01:37PM
(#10244941)
Haven't tried ADVI yet, but I think it is (likely) a great idea. In general automatic differentiation is a wildly powerful technique and is underutilized in computational science. Even in pure optimization problems people rarely leverage it to get J or H information as often as they probably should. I'll admit to not being super familiar with all the machinery under the hood in the ADVI paper [1]. It wasn't out when I started working on my current project.
Re:Stan: a probabilistic programming language
by proditus on Saturday September 19, 2015 @02:50PM
(#10245179)
do let me know how it goes if you do try ADVI.
we're also working on making it more robust to initialization and step-sizes. stay tuned.
Re:Stan: a probabilistic programming language
by mjw on Saturday September 19, 2015 @07:11PM
(#10245871)
AVDI is incredibly awesome. Thanks so much for this work. Really nice readable paper on it too.
As someone working with large datasets I think automating variational inference + SGD to work with a broad class of models is really the way forwards and a big force multiplier for machine learning.