Re:Time to Abolish "Statistical Significance"?
by nestorD on Saturday September 14, 2019 @05:49AM
(#20969820)
That's called the multiple comparaison problem in statistics and it is both well kown and compensated for in most studies (the hard part being not to have too mauch false negatives in an effort to keep the quantity of false positives constant) :
https://en.wikipedia.org/wiki/Multiple_comparisons_problem
It is very different to confirm a prediction (i.e. to look for a particle with a precisely predicted mass), than to fish for some unexpected signal in your data.
In some cases Economics could do the same: Looking for an effect in any age range could be post-processed to take into account that you are looking into many age groups.