News for hackers. Stuff that matters.
Re:Stan: a probabilistic programming language by probdist on Saturday September 19, 2015 @01:07PM (#10244842)
Stan is best viewed in my mind as a successor to BUGS (Bayesian Inference Using Gibbs Sampling) which more people may have heard of. In fact, there are lots of players in the probabilistic programming space now, personally I like the model of "Infer.NET" [1] from Microsoft Research, as I find variational and approximate variational inference a good solution to my problems and I like coding models in nearly C# more than in Stan format.

If you want to get started as fast as possible with computational Bayesian inference and don't need the performance and advanced features Stan, I'd recommend emcee [2] which is a lightweight python MCMC sampler.

Final recommendation to the HN crowd: Bayesian Data Analysis 3rd Edition [3] should be on your desk if you are thinking about using any of this software.

[1] http://research.microsoft.com/en-us/um/cambridge/projects/in...]

[2] http://dan.iel.fm/emcee/current/

[3] https://www.crcpress.com/Bayesian-Data-Analysis-Third-Editio...

Re:Stan: a probabilistic programming language by dustintran on Saturday September 19, 2015 @04:57PM (#10245521)
Hi, stan dev here. I think viewing Stan as a better BUGS is helpful but limiting.

The syntax is similar, but the class of models Stan fits is far more general. The class of algorithms we have available also goes beyond MCMC, e.g., variational inference, optimization, and interfaces to Stan exist on all primary programming languages. It's more helpful to think of Stan as its own probabilistic programming language, and arguably the biggest entity with the largest user base.

Re:Stan: a probabilistic programming language by thisisdave on Saturday September 19, 2015 @05:22PM (#10245576)
>the class of models Stan fits is far more general.

This is mostly true, but last time I checked, Stan still couldn't sample discrete variables like BUGS can. Stan can only fit models with discrete parameters (e.g. finite mixtures) if the programmer is smart enough to integrate them out.

Re:Stan: a probabilistic programming language by kristjankalm on Saturday September 19, 2015 @01:32PM (#10244927)
some useful books available as a PDF:

David Barber "Bayesian Reasoning and Machine Learning" http://web4.cs.ucl.ac.uk/staff/D.Barber/pmwiki/pmwiki.php?n=... -- Thorough and systematic, accompanied by a Matlab library

Marc Steyvers "Computational Statistics with Matlab" http://psiexp.ss.uci.edu/research/teachingP205C/205C.pdf -- Somewhat unfinished but very good hands-on guide for a total novice

David Draper "Bayesian Modeling, Inference and Prediction" https://users.soe.ucsc.edu/~draper/draper-BMIP-dec2005.pdf -- Quite good if you're already familiar with a field, might be slightly daunting for a beginner

and the classic, David MacKay's "Information Theory, Inference, and Learning Algorithms" http://www.inference.phy.cam.ac.uk/itprnn/book.html

Re:Stan: a probabilistic programming language by dewarrn1 on Saturday September 19, 2015 @03:59PM (#10245366)
Another endorsement for Gelman's books — they're some of the best instructional statistics work that I've read.
Re:Stan: a probabilistic programming language by eruditely on Saturday September 19, 2015 @09:56PM (#10246287)
You really think they're instructional? I would say it's sort of complete, but no means introductory.
Re:Stan: a probabilistic programming language by chmullig on Saturday September 19, 2015 @02:53PM (#10245188)
Stan is very much a successor to BUGS, although I'm not sure BUGS is better known outside certain social science research communities these days.
Re:Stan: a probabilistic programming language by lgas on Saturday September 19, 2015 @10:44PM (#10246392)
It's not.
Re:Stan: a probabilistic programming language by tel on Saturday September 19, 2015 @01:09PM (#10244848)
+1 on all of this (except I'm a fan of full MCMC, have you used ADVI?)
Re:Stan: a probabilistic programming language by probdist on Saturday September 19, 2015 @01:37PM (#10244941)
Haven't tried ADVI yet, but I think it is (likely) a great idea. In general automatic differentiation is a wildly powerful technique and is underutilized in computational science. Even in pure optimization problems people rarely leverage it to get J or H information as often as they probably should. I'll admit to not being super familiar with all the machinery under the hood in the ADVI paper [1]. It wasn't out when I started working on my current project.

[1] http://arxiv.org/abs/1506.03431

Re:Stan: a probabilistic programming language by proditus on Saturday September 19, 2015 @02:50PM (#10245179)
do let me know how it goes if you do try ADVI.

we're also working on making it more robust to initialization and step-sizes. stay tuned.

Re:Stan: a probabilistic programming language by mjw on Saturday September 19, 2015 @07:11PM (#10245871)
AVDI is incredibly awesome. Thanks so much for this work. Really nice readable paper on it too.

As someone working with large datasets I think automating variational inference + SGD to work with a broad class of models is really the way forwards and a big force multiplier for machine learning.