News for hackers. Stuff that matters.
Re:The Central Limit Theorem and Its Misuse by roenxi on Monday December 30, 2019 @05:17AM (#21912333)
The CLT can be used to justify modelling an effect with a normal distribution and raises questions about model assumptions when normal distributions don't appear where they should. The Normal Distribution is special because it is the highest-entropy distribution that is completely characterised by mean and standard deviation [0].

So what the CLT is saying is that, surprisingly, sums of i.i.d. random variables lose information from the individual variables quite quickly but reliably retain a little data about mean and second moment. Initially a given X_i has all sorts of information associated with it (higher order moments, other distribution characteristics, etc) that disappears as many X_i are summed together. All that is left is information about a mean and variance.

So say I have a situation where a large number of i.i.d. variables are going to be summed together. The CLT tells me that summing N of these variables together is going to be similar to summing together an equivalent number of normally distributed variables (!!). This is because the sum variable is equal in value to n * \bar{x}, but the CLT imposes a distribution on \frac{\bar{x}}{n}.

This justifies why the normal turns up everywhere in practical measurements. A lot of measurements (say, number of people at the beach) are probably really measures of a sum of random variables (maybe there is some non-normal variables that captures the chance a given person goes to the beach). So if the number of people at the beach turns out to be normally distributed (maybe I measure it each day for a few weeks) it isn't shocking. If the total number isn't normally distributed then that implies that there is no i.i.d. variable representing probability a given individual goes to the beach (eg, high correlation between the individual variables).

I didn't quite fail any of the statistics courses I've ever done, YMMV.

[0] https://en.wikipedia.org/wiki/Normal_distribution#Maximum_en...

There are no comments on this comment yet.