Re:What a debate about p-values shows about science
by sndean on Monday October 02, 2017 @08:19PM
(#15389055)
> However, if you have an experiment that pulls X from the system, watches Y fail, then readds X in a new way to recreate the system, you can be much more certain that X influences Y. That becomes a much desired, "elegant" experiment.
Yeah, in molecular biology this always seemed to be the most convincing evidence. Deleting a gene and then adding it back. Sometimes going a step further and overexpressing the gene. You can generally publish that data without any statistical analysis.
> The later experiment is often harder, and might not even be proposable until you've seen some statistical effect.
I think the better journals/reviewers can push back at this step, though: If you don't have a demonstrable mechanism, you can't publish. Don't let people stop at "...this gene is important (p < 0.005)." (I know not all things are as simple as gene deletion and complementation)
However, if you have an experiment that pulls X from the system, watches Y fail, then readds X in a new way to recreate the system, you can be much more certain that X influences Y. That becomes a much desired, "elegant" experiment.
The later experiment is often harder, and might not even be proposable until you've seen some statistical effect. However the scientific results from that experiment really don't hinge on statistical signifigances. The results are either binary in result, or better yet, contradictory - elucidating hitherto unknown variables (something a purely statistical result won't help with). And if mechanism is shown in multiple systems, with multiple techniques, the results can very quickly become near-definitive.