News for hackers. Stuff that matters.

standevbob

Re:Stan is a state-of-the-art platform for statistical modeling on Wednesday December 23, 2020 @03:38PM (#25521919)
Stan supports optimization (L-BFGS) to find (penalized) maximum likelihood or MAP estimates where they exist. Bayesian estimates are typically posterior means, which involve MCMC rather than optimization, and the result is usually far away from the maximum likelihood estimate in high dimensions. I wrote a case study with some simple examples here: https://mc-stan.org/users/documentation/case-studies/curse-d...

Adding new parameters scales as O(N^5/4) in HMC, whereas it scales as O(N^2) in Metropolis or Gibbs. It's quadrature that scales exponentially in dimension. There's also a constant factor for posterior correlation, which can get nasty. I regularly fit regressions for epidemiology or genomics or education with 10s or even 100s of thousands of parameters on my notebook with one core and no GPU.

MCMC or optimization can be sub-linear or super-linear in the data, depending on the statistical properties of the posterior. Some non-parametric models like Gaussian processes can be cubic in the data size, whereas regressions are often sub-linear (doubling the data doesn't double computation time) because posteriors are better behaved (more normal in the Gaussian sense) when there's more data and hence easier to explore in fewer log density and gradient evaluations.

Re:Stan is a state-of-the-art platform for statistical modeling on Wednesday December 23, 2020 @11:21AM (#25519180)
That's right---Stan doesn't have any online learning facilities. It's very hard to approximate posteriors and chain them, so we don't try.

If by "big data", we're talking about too big to fit in memory, that's right. Stan's fully in-memory. Compute can be distributed and GPU-powered for matrix ops, but all of the data and parameters and the core autodiff expression graph need to fit in memory.

For "medium data", Stan's adaptive Hamiltonian Monte Carlo sampling is much more efficient and scalable to complex models and higher dimensions than Gibbs or Metropolis. I'm fitting a Covid prevalence model using a custom trend-following and mean-reverting second-order autoregression model over 400 distinct regions with weekly data that has 5M data points and 10K parameters and adjusts for sensitivity and specificity of various tests taken. It fits in a single thread using MCMC in 24 hours or so, but we can fit the model with variational inference in a couple minutes. Although variational inference often produces reasonable point estimates in bigger data settings, it doesn't reasonably quantify uncertainty. I'm also working on a genomics model for differential expression of splice variants that involves 120K measurements and just as many parameters to deal with overdispersion of biological replicates in a control and treatment group. We're using variational inference and it fits in a couple minutes for the comparitiver event probabilities we need to estimate.

Re:Stan is a state-of-the-art platform for statistical modeling on Wednesday December 23, 2020 @11:08AM (#25519024)
This is the main reason that people use Stan---squeezing as much info out of your data as possible. That and the ability to write custom models for these situations.

There are hundreds of different applications of Stan across the physical, biologial, and social sciences, as well as in finance, education, sports analytics, actuarial sciences, transportation planning, all sorts of material and chemical and civil engineering, clinical trials and pharmacometrics, etc. etc. It's most popular in fields like ecology and epidemiology where Bayesian methods are already popular. For instance, many of the Covid models (like the one for NY state) are being built with Stan. All four baseball teams in the semifinals (LCS) use Stan for analytics, for example. Google and Facebook use Stan for ad attribution and resource allocation. It's been used for models of neutrino mass and models of galactic mass, models of supernovas, and it's even used in the LIGO gravitational wave experiments.

Re:Stan is a state-of-the-art platform for statistical modeling on Wednesday December 23, 2020 @10:59AM (#25518929)
Stan requires models to be coded in the Stan language, which is a simple imperative language that's like MATLAB with explicit data types. This is the same as was done in Stan's predecessors, BUGS and JAGS.

A Stan program can be run in any of our interfaces in Python, Julia, R, MATLAB, Stata, etc. But you can't mix any of those languages into a Stan program.

The C++ toolchain is required because Stan transpiles its programs to C++, then compiles those against the Stan math librarym, which does autodiff. But you don't need to write any C++ to use Stan, just to develop extensions for it.

Re:Stan is a state-of-the-art platform for statistical modeling on Wednesday December 23, 2020 @10:54AM (#25518860)
Stan provides both frequentist inference (penalized maximum likelihood with bootstrapped confidence intervals) and Bayesian inference (MCMC sampling or approximate variational) inference.

As currymj says, the differential equations (same for all the linear algebra solvers like eigendecomposition) can be used in defining likelihoods for either Bayesian or frequentist estimation. Same for all of our linear algebra operations and special functions.

Not every model that can be programmed in Stan has a well-defined MLE or proper posterior. Standard hierarchical/multilevel models don't have MLEs, even with standard shrinkage. Bayesian models with improper priors and no data wind up with improper posteriors, etc.

Having said all that, almost all of the use of Stan is for Bayesian inference.

Re:Stan is a state-of-the-art platform for statistical modeling on Wednesday December 23, 2020 @10:44AM (#25518725)
Yes, we regularly use Stan's MCMC to fit relatively simple time-series regression models or item-response theory type models with 10^5 parameters and 10^6 rows of data on a desktop computer. It can take a day, though. It's much faster with variational inference, but that can be less stable and it doesn't give you the same uncertainty quantification because of the way the KL-divergence is ordered in the objective.

Stan can parallelize multiple chains and it can parallelize the density/gradient calculations in a single chain. But for the latter to be efficient, the chunks being parallelized need to be compute intensive, like you might get in a pharmacometric compartment model where you might have to solve a bunch of differential equations for each of thousands of patients in a clinical trial.