Re:Noted study in psychology fails to replicate, crumbles wi...
by Retric on Sunday August 22, 2021 @07:29AM
(#28264599)
Being possible to replicate isn’t the same as every experiment getting identical results. The point of doing statistics is to separate signal from noise, but it also means ignoring results where the signal wasn’t strong enough because your sample size was to small etc.
Suppose your testing a weight loss drug and in the first experiment subjects lost 5 pounds and the second they only lost 3. On the surface the first study replicates fine, except the second study has a weaker signal so it might not past the statistical threshold chosen before the study. This is fairly common issue as you should expect random noise to amplify a signal in about half of experiments and more than half of all published research.
Suppose your testing a weight loss drug and in the first experiment subjects lost 5 pounds and the second they only lost 3. On the surface the first study replicates fine, except the second study has a weaker signal so it might not past the statistical threshold chosen before the study. This is fairly common issue as you should expect random noise to amplify a signal in about half of experiments and more than half of all published research.