I will give you $999 if it's just noise in exchange for $1 if it's not.

Looks like some people didn't like you putting the concept in more accessible terms. "Three sigma" has a real interpretation, it's not just a little score where you pull out LaTeX when it reaches 6.

Of course, to perform any valid statistical reasoning, you need to consider the prior. The standard model has held up very well, and we do a lot of experiments-- any of which has a small chance of popping out a result that looks highly significant by chance. There's a reason why 6 sigma is the line to take things super seriously (in fundamental physics).

But LaTeX seems to take a whole lot longer to run when it's 6 sigma than when it's 3.

You are joking, right? The experiments around the LHC have produced a handful of three-sigma "non-standard-model" results already. So far they have all gone away.

This is not a bad thing, by the way - just an inevitable consequence of doing many searches.

Shouldn’t there just be around 1 in 1000 experiments the state reach 3 sigma for a result which then turn out to be noise?

I guess that only applies if there are no mistakes or biases at all?

You're not seeing all the searches that didn't pan out.

Yes. I think it's a joke.

As in 3-sigma ~= 1/1000

Yes but he brings up a good point. They are more likely to talk about a 3 sigma result that challenges the standard model than one that doesn’t yield anything special. So given you know a 3 sigma result is being discussed, the probability that it’s a false result at higher sigma is more than 1/1000.

Break-even for three sigma would be 997:1, so the parent should actually take this bet.

The result was past three sigma, not exactly at three sigma. But good point, because if it was exactly at three sigma I would have snookered myself. :)

The model indicates 1000:1, but it may not capture all aspects (i.e. correlations) in the system. So what seems like excessive sigmas is also margin for those unknown-unknowns.

Hence the qualifier "if it's just noise," ruling out other falsifications.

Plenty of 3 sigma results have come and gone, (there's obviously an xkcd for this about doing a whole lot of experiments). It's certainly something to watch and I would personally be stoked for some new physics.

I only posted because it was the first thing I wanted to know, and I had to dig around in the article for a bit before I found it at the end.

My mentor once said "3 sigma effects happen in physics a lot more often than they should." I've since come to realize that we should do a multiple-hypothesis testing correction on all such results, where the number of hypotheses tested is approximately equal to the number of Ph.D.s granted in particle physics.

How about 999 subscriptions to Penthouse in exchange for 1 subscription to Private Eye?