News for hackers. Stuff that matters.
Re:How to Avoid Idea Averaging by GFK_of_xmaspast on Tuesday January 05, 2016 @09:11PM (#10847938)
> Assuming you gather enough data, your posterior distribution will eventually approximate a delta function centered around that one true model. This is also what happens when you do BMA or gaussian mixtures.

What happens when your data is actually distributed according to a mixture of gaussians (c.f. Pearson's crabs).

Re:How to Avoid Idea Averaging by yummyfajitas on Tuesday January 05, 2016 @09:48PM (#10848103)
In a situation like that, you have a space of models M and a space of populations P. The state space is M x P, and the model M generates the population P (probabilistically). In Pearson's crab example, there are some models where an individual crab can come from one or more gaussians.

The generative model (first choose gaussian, then draw from it) is the specific model. As you gather a sufficiently large sample from the population P, eventually you'll converge to a delta-function on the space of M.

So in Pearson's crab example, given enough data, you'll eventually either converge to the single model of 2 gaussian crab species, or you'll converge to a 3 gaussian crab species model, or you'll converge to a weibull single species model (assuming you put all 3 of those models into your prior).