Re:Understanding Statistical Power and Significance Testing
by mlthoughts2018 on Monday November 09, 2020 @03:02PM
(#25039205)
In my org we frequently run Bayesian regression fitting on datasets of millions of samples.
The key is: don’t use pymc or stan for large data, just actually write your own MCMC code and write log likelihoods for your own models. It’s very easy and very fast, even in Python.
We do still use pymc and stan for other, smaller modeling tasks.
Re:Understanding Statistical Power and Significance Testing
by disgruntledphd2 on Tuesday November 10, 2020 @04:33AM
(#25044150)
Yeah, but it's not worth it for my purposes. Given the kinds of data and the wide variety of problems we deal with, it would be an investment of too much resources relative to the rewards.
I actually think that many of the people who promote Bayes everywhere have never tried to run a simple Stan regression on 100k+ data-points (pro-tip: sample, then sample some more, give up as it's taking too long).
That being said, from a philosophical point of view, Bayes is definitely the way I think.