http://blogs.scientificamerican.com/observations/2012/07/17/...

Edit: oh, no, actually I think you're right :). I thought it was referring to the location of the test statistic in its null distribution. But it seems it's a scale for measuring p-values. This explains it clearly:

What does "five sigma" mean? It means that the results would occur by chance alone as rarely as a value sampled from a Gaussian distribution would be five standard deviations from the mean.

http://www.graphpad.com/www/data-analysis-resource-center/bl...

By the way, this same conversation took place after the LHC Higgs anouncement -- the same five-sigma standard for discovery, and the same detailed discussion of what that means.

> But it seems it's a scale for measuring p-values.

Yes, and that was a point I made in my reply to the OP -- that the context assumed an association with p-values, which in turn assume a normal distribution.

The only way that the Normal distribution comes into play is that physicists are measuring the smallness of their p-values by stating a number of standard deviations from the Normal mean that would have the same p-value.

I'm finding this discussion helpful by the way. I have worked in applied statistics but not in any fields which use this "sigma" scale or would talk about "sigma values".

No, not unless a p-value is expressed in terms of sigma as in this case and similar ones. In this case, and commonly in experimental physics, there's a relationship between n-sigma (usually 3σ or 5σ in different circumstances) and how a p-value is acquired from a sigma expression. The p-value is acquired from a sigma value like this:

http://i.imgur.com/pcjr6cN.gif

My point? In experimental physics there's a connection between (a) an expression including an integer and "sigma", (b) a resulting, widely quoted numerical value, and (c) the method for converting one to the other, using a Gaussian distribution as shown.

http://physicsbuzz.physicscentral.com/2012/07/does-5-sigma-d...

Quote: "But what does a 5-sigma result mean, and why do particle physicists use this as a benchmark for discoveries?

To answer these questions, we'll have to look at one of the statistician's oldest friends and C-student's worst enemies: the normal distribution or bell curve."

Couldn't have said it better myself.

> The only way that the Normal distribution comes into play is that physicists are measuring the smallness of their p-values by stating a number of standard deviations from the Normal mean that would have the same p-value.

Hmm. Yes, that's right. That's why I replied as I did in my original post.

In physics, a sigma value maps to a p-value, and that relationship is most often defined with respect to a normal distribution. Therefore, in most cases, to go from a sigma value to a p-value, one performs this integral:

http://i.imgur.com/YhC302b.gif

Specifically, the above definite integral, when performed with arguments of 5 and +oo, yields the often-quoted one-tailed p-value for "5 sigma", which we can get here as well:

https://www.wolframalpha.com/input/?i=5+sigma

It seems Wolfram Alpha makes the same default assumption I do: a normal distribution. If I weren't answering an inquiry from someone who wanted the clearest possible answer, I might have replied differently.

I'm absolutely correct in what I said.

But it appears that you are correct that that part of physics makes some Gaussian assumptions and, then, has some conventions based on those assumptions. In this case, apparently physics is not making its mathematical assumptions clear and explicit and is doing sloppy writing. For more detail, see my longer explanation in this thread

https://news.ycombinator.com/item?id=7421505

Maybe the situation is a little like getting from a little French restaurant the recipe for French salad dressing, sauce vinaigrette, making it at home, and concluding it tasted better in the little French restaurant. Hmm .... But the French restaurant did something not in the recipe -- took a large clove of garlic, peeled it, cut it in half, and wiped the salad bowl with the cut surface of the garlic! The recipe didn't mention that!

"In statistics, σ refers to an area under the normal distribution defined in terms of standard deviations (1σ = 1 standard deviation)."

He was just stating that the standard deviation is a general concept of spread, that any distribution has, and not just the normal one.

> "In statistics, σ refers to an area under the normal distribution defined in terms of standard deviations (1σ = 1 standard deviation)."

Not only is that not idiotic, that's the default definition in a statistical context (see below). Anyone can argue that σ is just another Greek letter with no special significance, but that require one to ignore the context in which the term is used.

Link: http://en.wikipedia.org/wiki/Standard_deviation

Quote: "In statistics and probability theory, the standard deviation (SD) (represented by the Greek letter sigma, σ) shows how much variation or dispersion from the average exists."

> He was just stating that the standard deviation is a general concept of spread, that any distribution has, and not just the normal one.

Again, this disregards context. When nontechnical people ask what the significance of 5σ is to scientific statistical analysis in physics (which is how this thread got started), there is precisely one answer.

No, you're quite wrong about that. greycat is correct in his/her corrections of what you're saying. In statistics sigma is used to represent one of two things:

- a parameter of a probability distribution, typically one which influences the spread of the distribution

- a measure of dispersion in an actual data set, which may be an estimator of a parameter in a probability distribution

Neither of those things necessarily involve the Gaussian density.

In physics it seems that "sigma values" are used as a scale to measure p-values, so that instead of saying 0.0000003, they can just say 5σ. But the critical point here, which your comments seem to be missing, is that there is no distributional assumption being made; there is no implication that the Normal distribution describes any data-generating process, merely that the probability of an equal or more extreme value of a test statistic under some model, is the same as the probability of observing a value more than 5 standard deviations from the mean under a Gaussian model.

> No, you're quite wrong about that.

It is the default, actually. There are plenty of exceptions to the default, but it certainly is common in the context of experimental physics, the present context.

http://physicsbuzz.physicscentral.com/2012/07/does-5-sigma-d...

Quote: "But what does a 5-sigma result mean, and why do particle physicists use this as a benchmark for discoveries?

To answer these questions, we'll have to look at one of the statistician's oldest friends and C-student's worst enemies: the normal distribution or bell curve."

> ... your comments seem to be missing, is that there is no distributional assumption being made ...

Do read some experimental physics -- see what assumptions are made. Here is how a physicist maps a sigma value to a p-value:

http://i.imgur.com/pcjr6cN.gif

If this wasn't a discussion of the analysis of the outcome of a physics experiment, I would be more likely to accept these digressions.

lutusp, the way in which you are using the word "assumption" carries a very high risk that people will misunderstand you. The critical point here is that the physicists are nowhere using the Normal distribution as a modeling assumption. They are not suggesting that the Normal distribution is a reasonable model for any real data generating process in their problem domain. They are simply using it as a scale, like Celsius of Fahrenheit. There's a crucial philosophical distinction there that, even if you get, your readers will not.

In statistics, σ refers to an area under the normal distribution defined in terms of standard deviations (1σ = 1 standard deviation).

In statistics and probability theory, the standard deviation (SD) (represented by the Greek letter sigma, σ) shows how much variation or dispersion from the average exists.

I think you're missing the difference.

http://i.imgur.com/pcjr6cN.gif

Note that the unit normal distribution is the underlying context.