Re:Scientists rise up against statistical significance
by abecedarius on Wednesday March 20, 2019 @09:40PM
(#19448624)
Hippocratic medicine lasted well into the 19th century, centuries after the scientific revolution. There'd been critics correctly calling it an intellectual fraud before then. You could've taken this as proof that no force on Earth could drag medicine into modernity, but it did sort of happen, as it became public, common knowledge that doctors were harming more people than they helped. They did start cleaning up their act (literally) though it took a long time and I think they're still collectively irrational about chronic conditions.
I hope we aren't worse at reform than they were in the 1800s.
The problem is that with all its problems, statistical significance provides one major advantage over more meaningful methods: it provides pre-canned tests and a number (.05, .01, etc) that you need to 'beat'. The pre-canned-ness/standardization provides benchmarks for publication.
I once worked in a computational genomics lab. We got a paper into PNAS by running fisher-exact test on huge (N=100000+) dataset, ranked the p-values, got the lowest p-values, and reported those as findings. There's so much wrong with that procedure its not even funny.