News for hackers. Stuff that matters.
Re:Intriguing new result from the LHCb experiment at CERN by ramraj07 on Monday March 29, 2021 @04:56AM (#26619003)
Well put, but I’ve always felt a middle ground existed where a knowledgeable scientist can make a very educated guess with much lesser statistical significance as long as the experiment is thought through, you are aware of the noise models and alternative explanations have been explored and mostly ruled out. Of course for publication you better whip your ass to whatever sigma they demand, but more often than not there either cause for optimism or pessimism ways before that point.
Re:Intriguing new result from the LHCb experiment at CERN by marcosdumay on Monday March 29, 2021 @09:15AM (#26621042)
Not really. The amount of confidence you have grows exponentially with the observed deviation, and that last one grows kinda proportionally to the amount of data you have (if the hypothesis is real, of course).

So there is some fine adjustment possible, but it's very small on the experiments where you need a lot of confidence. There is more margin to judge low non-significant results at the low confidence end, as exponentials grow slowly when they are small.