Re:Reduced cancer mortality with daily Vitamin D intake
by anonymouskimmer on Tuesday May 09, 2023 @11:23PM
(#35883276)
> They intended to demonstrate that you need a _reason_ to want to split
No, no, no. FFS no! Sure this is a good thing to do if you are trying to prove a hypothesis. But you you are trying to explore for truly novel and unexpected linkages p-hack to your heart's content, form hypothesis, and then do follow up studies to see whether there's really something there!
Ignoring possibilities is bad exploratory science. And, as a reader, it's quite annoying to read old studies only to find that the author's didn't bother splitting on the parameter you are currently interested in.
Split them all, if only in the supplementary data, and let future studies sort them out.
I don't know how to put it less bluntly: you're incorrect.
> But sure, the intent they tried to demonstrate (take all sub-group analyses with a grain of salt)
That's not what they intended to demonstrate. They intended to demonstrate that you need a _reason_ to want to split, and that reason needs to be given _ahead_ of the analysis. If you see the results _and then choose_ a new data analysis (that includes new subgroup analyses), your procedure is no longer statistically sound.
This is a specifically bad statistical practice called HARKing https://en.wikipedia.org/wiki/HARKing