News for hackers. Stuff that matters.
Re:Markov Chain Monte Carlo (MCMC) Sampling, Part 1: The Bas... by michaericalribo on Wednesday October 28, 2020 @09:34PM (#24926158)
The Gibbs sampler is right at the top of my personal list of the greatest technical achievements in statistics. Not only is the asymptotic guarantee of producing a sample from the target distribution a neat and elegant result, but it also opened up a really rich world of probabilistic modeling. From a practical / ML perspective, variational methods are more efficient for getting decent estimates of complex models. And there are a lot of devilish details to MCMC sampling, and the real world _never_ goes to infinity. Still, the sheer theoretical fact that an imperfect, simulation-based method can produce valid statistical estimates is just awesome.
Re:Markov Chain Monte Carlo (MCMC) Sampling, Part 1: The Bas... by frodetb on Thursday October 29, 2020 @04:42AM (#24928435)
I wrote my masters on using simulated annealing to solve a particular optimisation problem. In a sense, the Gibbs sampler lies at its core, and I really have to agree with you in what you say. At the time I did not have time to think too deeply or too much about it, but I have since really been increasingly awestruck with it, with thermodynamics and statistical mechanics in general, and analogies that can be drawn to all kinds of disciplines.
Re:Markov Chain Monte Carlo (MCMC) Sampling, Part 1: The Bas... by tetris11 on Thursday October 29, 2020 @07:22AM (#24929296)
In bioinformatics, MCMC was used often linkage analysis to sample a "genotype space" of a true inheritance pattern of multiple related individuals to find the location and transmission pathway of disease causing variants.

Classical genetics and statistics were very tightly linked in this regard due to scarcity of data at the time (pre-2000).

Now that the field has exploded with big data, such statistics are no longer so neccesary to answer the same problems, and a lot of the early mathematical elegance associated with the field has given way to brute-force filtering methods.

Re:Markov Chain Monte Carlo (MCMC) Sampling, Part 1: The Bas... by wheresmycraisin on Thursday October 29, 2020 @03:01AM (#24928002)
I appreciate the theoretical aspects of it as well -- however, I could never get over the notion that bayesians need to use frequentist statistics to judge whether the chain has sufficiently converged... isn't that defeating the entire point?
Re:Markov Chain Monte Carlo (MCMC) Sampling, Part 1: The Bas... by soVeryTired on Thursday October 29, 2020 @06:09AM (#24928839)
Interesting point, but I think the academic culture has taken a pragmatic shift: there aren't many 'pure' bayesians or frequentists left anymore, and the boundary between the two viewpoints has softened. Stats has more of a 'shut up and calculate' mentality about it these days.
Re:Markov Chain Monte Carlo (MCMC) Sampling, Part 1: The Bas... by sgt101 on Thursday October 29, 2020 @06:29AM (#24928963)
>Stats has more of a 'shut up and calculate' mentality about it these days.

I'm happy that the "pure this pure that" fundamentalism has faded into retirement... but shut up and calculate has problems - especially around causality. The "blessings of multiple causes" paper for example.

Re:Markov Chain Monte Carlo (MCMC) Sampling, Part 1: The Bas... by wheresmycraisin on Friday October 30, 2020 @07:45PM (#24947898)
Then why bother pretending to calculate a posterior distribution when you can do a MLE estimator with regularization or something?