Re:Psychologists Welcome Analysis Casting Doubt on Studies
by analog31 on Saturday August 29, 2015 @10:32AM
(#10140080)
I know people in the life sciences who admit to frustration over the pressure to publish and the abuse of statistics.
But I don't know if statistics is the entire problem. It seems to me that a feature of good work in the "hard" sciences is that a problem will be approached from a bunch of different angles, so that the result doesn't really depend solely on statistics.
Synthetic chemists have to perform multiple orthogonal tests in order to announce a new compound, such as IR, NMR, X-Ray crystallography, and elemental analysis, on top of describing a mechanism for the compound's formation and performing reactions with the compound if possible.
Traditionally, to prove discovery of a new element, it wasn't good enough to see a blip on a mass spec. You had to do chemistry with it.
In my view a problem with data driven science is that you can do statistics without even knowing what you're actually studying. But if you don't know what you're studying, then you can't attack a problem from multiple angles because you don't even know if your tests are all measuring the same thing. You also have no tools to ask the hard questions about whether the results really make sense.
But I don't know if statistics is the entire problem. It seems to me that a feature of good work in the "hard" sciences is that a problem will be approached from a bunch of different angles, so that the result doesn't really depend solely on statistics.
Synthetic chemists have to perform multiple orthogonal tests in order to announce a new compound, such as IR, NMR, X-Ray crystallography, and elemental analysis, on top of describing a mechanism for the compound's formation and performing reactions with the compound if possible.
Traditionally, to prove discovery of a new element, it wasn't good enough to see a blip on a mass spec. You had to do chemistry with it.
In my view a problem with data driven science is that you can do statistics without even knowing what you're actually studying. But if you don't know what you're studying, then you can't attack a problem from multiple angles because you don't even know if your tests are all measuring the same thing. You also have no tools to ask the hard questions about whether the results really make sense.