News for hackers. Stuff that matters.
Re:Stan: a probabilistic programming language by nextos on Saturday September 19, 2015 @02:05PM (#10245030)
You can easily encode any generative sequence model in e.g. Church (Scheme-based), Anglican (Clojure-based) or PyMC (Python-based). These are Turing-complete, so they can model any sampleable (computable) distribution.

See http://forestdb.org/ for some examples. E.g. an infinite HMM.

I have no experience with Stan. There was some controversy on whether it is Turing-complete, but I cannot elaborate on that.

The obvious disadvantage of using a language that is too expressive for something as simple as an HMM is that you loose the nice performance guarantees given by all specialized algorithms such as Viterbi. Theoretically, they can achieve good efficiency by performing program transformations (e.g. with abstract interpretation). But in practice, we are still a bit far from that.

A nice trick is to implement your generative model in something very efficient (e.g. Probabilisitic C or something GPU-based). You can then forget the burden of having to encode your own sampling procedure, but at the same time inference might be tractable.

Re:Stan: a probabilistic programming language by mjn on Saturday September 19, 2015 @02:39PM (#10245144)
There's some research on automatically deriving efficient algorithms for classes of distributions, e.g. getting a specialized EM-like algorithm for anything where EM-like algorithms are applicable: http://machinelearning.wustl.edu/mlpapers/paper_files/AA24.p...

But obviously that doesn't cover classes as big as "anything expressible in Church", so there's a pretty big gap. It's possible that languages like Church could use techniques like that as part of an optimizing compiler that recognizes specializable patterns, the way regular compilers do loop transformations and auto-vectorization and such. But, like auto-vectorization, not that easy to do.

Re:Stan: a probabilistic programming language by nextos on Saturday September 19, 2015 @02:43PM (#10245158)
Sure, I mentioned abstract interpretation, but as you say all optimizing compiler techniques might be applicable.