Re:HackPPL: A universal probabilistic programming language
by cshenton on Saturday October 19, 2019 @11:40AM
(#21299038)
Not sure why one would use this over Uber’s Pyro, which is built in python on top of Pytorch and has a similar feature set.
Also worth pointing out that, like all current gen PPLs, the word “Universal” is highly misleading, as only variables that are the result of a sample statement can be observed (since HMC, IS, and others require closed form conditional likelihood’s for observed variables). This rules out use cases such as observing noiseless aggregates of noisy signals (any workaround is a fudge and will put strain on the inference algorithms capacity for search).
Only a PPL with support for likelihood free inference techniques (few of which work very well as of today) could be considered truly “Universal”.
Re:HackPPL: A universal probabilistic programming language
by doctorpangloss on Saturday October 19, 2019 @12:48PM
(#21299368)
Pyro is pretty wonky too. Something like Gen, which is still a big research project, has a lot of hope for the sort of universal behaviour you're describing.
In practice, so many ordinary programs in ordinary languages use heap-allocated lists with runtime-specified lifetimes as intermediate values and return values. That sounds wonky, but I'm saying there's a lot of "businessLogic(X) -> variable length, nondeterministic on X length List of Y" functions, which is ill suited for PPL modeling. On the other hand, neural networks, which really work on binary images of inputs and outputs, handle those situations surprisingly well. That's what you're really competing with.
Re:HackPPL: A universal probabilistic programming language
by marmaduke on Saturday October 19, 2019 @04:50PM
(#21300685)
I got Pyro to perform almost as well as Stan on a complex model under variational inference, with the upside that I could refactor it all into classes and integrate with a larger Python framework, all while making use of GPUs. Could it be any better? What do you see as wonky?
Also worth pointing out that, like all current gen PPLs, the word “Universal” is highly misleading, as only variables that are the result of a sample statement can be observed (since HMC, IS, and others require closed form conditional likelihood’s for observed variables). This rules out use cases such as observing noiseless aggregates of noisy signals (any workaround is a fudge and will put strain on the inference algorithms capacity for search).
Only a PPL with support for likelihood free inference techniques (few of which work very well as of today) could be considered truly “Universal”.
In practice, so many ordinary programs in ordinary languages use heap-allocated lists with runtime-specified lifetimes as intermediate values and return values. That sounds wonky, but I'm saying there's a lot of "businessLogic(X) -> variable length, nondeterministic on X length List of Y" functions, which is ill suited for PPL modeling. On the other hand, neural networks, which really work on binary images of inputs and outputs, handle those situations surprisingly well. That's what you're really competing with.