News for hackers. Stuff that matters.
Re:Why isn't everything normally distributed? by logicallee on Thursday July 20, 2017 @06:46PM (#14816877)
that's really weird. I'm not an astronomer so I thought "hmmm, what's the first thing I can think of in astronomy."

Well, how about the magnitude of stars. So, is the magnitude of stars normally distributed? I googled "distribution magnitude of stars" and got images (by clicking on images tab) that seem pretty normally distributed to me. Aren't they?

Re:Why isn't everything normally distributed? by jackpirate on Thursday July 20, 2017 @07:25PM (#14817030)
I think you're being downvoted a bit unfairly. What most non-statisticians don't realize is that some of the most "unnormal" distributions actually look pretty normally distributed, and some of the most ugly looking distributions actually behave like a normal distribution. When you look at a distribution's pdf, you're only looking at the "center" of the distribution. But what statisticians (usually) care about are the tails of a distribution.

Consider the following example: The [Cauchy distribution](https://en.wikipedia.org/wiki/Cauchy_distribution) is a pathological example of a distribution with very heavy tails. Heavier than the power law distributions meantioned in the GP. So heavy, the distribution doesn't even have a mean value. But if you just look at the pdf, it looks almost exactly like a gaussian.

Re:Why isn't everything normally distributed? by CuriouslyC on Friday July 21, 2017 @11:23AM (#14821249)
It is pretty easy to tell a Cauchy or a low k Student's T apart from a Gaussian. Just look at your outliers. One thing a lot of people miss is that a normal distribution with outliers _IS_ a long tail distribution.

Too bad inference with long tailed distributions is so hard.

Re:Why isn't everything normally distributed? by logicallee on Thursday July 20, 2017 @07:41PM (#14817105)
thanks! Very interesting. Is the standard deviation still useful for some of the other distributions you mentioned, that look vaguely Gaussian?
Re:Why isn't everything normally distributed? by R_haterade on Thursday July 20, 2017 @08:17PM (#14817242)
You're referring to the square root of the second moment about the mean. The short answer is yes, it's always useful, provided you have an estimator.

Check out Chebyshev's inequality.

Just don't start talking about it to your pointy-haired boss. You'll get fired.

Re:Why isn't everything normally distributed? by logicallee on Friday July 21, 2017 @01:52AM (#14818355)
thanks. but what does your last line, about the boss mean?
Re:Why isn't everything normally distributed? by R_haterade on Monday July 24, 2017 @03:53PM (#14842049)
That this sort of knowledge is often uninteresting to non-technical types.

I'm pretty jaded at this point.

Re:Why isn't everything normally distributed? by dbaupp on Thursday July 20, 2017 @09:14PM (#14817412)
For the specific example of the Cauchy distribution, the standard deviation isn't defined.
Re:Why isn't everything normally distributed? by thanatropism on Thursday July 20, 2017 @07:49PM (#14817135)
The standard deviation is defined as

E[(X-EX)^2]

If EX = infinity but X is finite with prob > 0, the standard deviation will be infinite too.

Re:Why isn't everything normally distributed? by felippee on Thursday July 20, 2017 @06:59PM (#14816930)
You mean like this one?

http://www.astro.yale.edu/astrom/spmcat/histov.html

Not really. Plus keep in mind magnitude is a logarithmic measure. Things get skewed when you take logarithms. Also if you look for example at star sizes:

http://spiff.rit.edu/classes/phys440/lectures/size/diam_hist...

(from http://spiff.rit.edu/classes/phys440/lectures/size/size.html).

The distribution is certainly not gaussian, and very much likely heavy tail.

Re:Why isn't everything normally distributed? by teraflop on Thursday July 20, 2017 @07:25PM (#14817028)
Not to mention that the distributions from three different sky surveys (in your first link) look dramatically different. Which means the histograms are saying at least as much about the measurement process as they are about what's actually out there in the universe.
Re:Why isn't everything normally distributed? by privong on Thursday July 20, 2017 @09:23PM (#14817451)
To add to that, the sharp cutoff at faint (large number) magnitudes is almost certainly a reflection of the fact that astronomical surveys and images have a flux limit, below which objects cannot be individually detected.
Re:Why isn't everything normally distributed? by pishpash on Thursday July 20, 2017 @06:52PM (#14816906)
First sentence on Wikipedia: "In astronomy, magnitude is a logarithmic measure of the brightness of an object..."
Re:Why isn't everything normally distributed? by privong on Thursday July 20, 2017 @07:09PM (#14816964)
> First sentence on Wikipedia: "In astronomy, magnitude is a logarithmic measure of the brightness of an object..."

That doesn't tell you whether the magnitudes are normally distributed though. That just tells you how the magnitude is computed from an object's observed/measured flux.

Re:Why isn't everything normally distributed? by pishpash on Thursday July 20, 2017 @11:01PM (#14817743)
I'm explaining why magnitude looking normally distributed doesn't contradict the parent.