Re:The Central Limit Theorem and Its Misuse
by roenxi on Monday December 30, 2019 @05:17AM
(#21912333)
The CLT can be used to justify modelling an effect with a normal distribution and raises questions about model assumptions when normal distributions don't appear where they should. The Normal Distribution is special because it is the highest-entropy distribution that is completely characterised by mean and standard deviation [0].
So what the CLT is saying is that, surprisingly, sums of i.i.d. random variables lose information from the individual variables quite quickly but reliably retain a little data about mean and second moment. Initially a given X_i has all sorts of information associated with it (higher order moments, other distribution characteristics, etc) that disappears as many X_i are summed together. All that is left is information about a mean and variance.
So say I have a situation where a large number of i.i.d. variables are going to be summed together. The CLT tells me that summing N of these variables together is going to be similar to summing together an equivalent number of normally distributed variables (!!). This is because the sum variable is equal in value to n * \bar{x}, but the CLT imposes a distribution on \frac{\bar{x}}{n}.
This justifies why the normal turns up everywhere in practical measurements. A lot of measurements (say, number of people at the beach) are probably really measures of a sum of random variables (maybe there is some non-normal variables that captures the chance a given person goes to the beach). So if the number of people at the beach turns out to be normally distributed (maybe I measure it each day for a few weeks) it isn't shocking. If the total number isn't normally distributed then that implies that there is no i.i.d. variable representing probability a given individual goes to the beach (eg, high correlation between the individual variables).
I didn't quite fail any of the statistics courses I've ever done, YMMV.
So what the CLT is saying is that, surprisingly, sums of i.i.d. random variables lose information from the individual variables quite quickly but reliably retain a little data about mean and second moment. Initially a given X_i has all sorts of information associated with it (higher order moments, other distribution characteristics, etc) that disappears as many X_i are summed together. All that is left is information about a mean and variance.
So say I have a situation where a large number of i.i.d. variables are going to be summed together. The CLT tells me that summing N of these variables together is going to be similar to summing together an equivalent number of normally distributed variables (!!). This is because the sum variable is equal in value to n * \bar{x}, but the CLT imposes a distribution on \frac{\bar{x}}{n}.
This justifies why the normal turns up everywhere in practical measurements. A lot of measurements (say, number of people at the beach) are probably really measures of a sum of random variables (maybe there is some non-normal variables that captures the chance a given person goes to the beach). So if the number of people at the beach turns out to be normally distributed (maybe I measure it each day for a few weeks) it isn't shocking. If the total number isn't normally distributed then that implies that there is no i.i.d. variable representing probability a given individual goes to the beach (eg, high correlation between the individual variables).
I didn't quite fail any of the statistics courses I've ever done, YMMV.
[0] https://en.wikipedia.org/wiki/Normal_distribution#Maximum_en...