Re:Effect size is significantly more important than statisti...
by candiodari on Tuesday September 14, 2021 @07:01PM
(#28532686)
You do realize there is a million negative results for every one positive result? This is equally easy to game, maybe easier.
[ Parent
| on HN ]
Re:Effect size is significantly more important than statisti...
by naasking on Tuesday September 14, 2021 @07:37PM
(#28533044)
Yes, and knowing what's been tried and what has failed is important.
Re:Effect size is significantly more important than statisti...
by TOMDM on Tuesday September 14, 2021 @10:00PM
(#28534312)
I think what's being pointed out is that "researchers" could pump out hundreds of easy to test negatives every day if a negative result was just as incentivised.
I do agree though, negatives are just as important when the intent is to prove/disprove a meaningful hypothosis.
Re:Effect size is significantly more important than statisti...
by kuhewa on Wednesday September 15, 2021 @01:28AM
(#28535564)
A negative result won't make a career. I don't think there's much danger when requiring negative results going onto a repository of over incentivising negative results. You can't mandate Nature or Cell publishes negative results.
Re:Effect size is significantly more important than statisti...
by hprotagonist on Tuesday September 14, 2021 @10:01PM
(#28534316)
we tried using 0.1 mL, it didn't work
we tried using 0.11 mL, it didn't work
we tried using 0.12 mL, it didn't work
we tried using 0.13 mL, it didn't work
Re:Effect size is significantly more important than statisti...
by harry8 on Wednesday September 15, 2021 @03:28AM
(#28536269)
we tried using 0.10 mL, it didn't work
we tried using 0.11 mL, it didn't work
we tried using 0.13 mL, it didn't work
we tried using 0.15 mL, it didn't work
we tried using 0.17 mL, it didn't work
we tried using 0.16 mL, it didn't work
we tried using 0.18 mL, it didn't work
we tried using 0.20 mL, it didn't work
we tried using 0.14 mL, it didn't work
we tried using 0.12 mL, it worked so we published
Do you want to know the ones that "didn't work" existed? Or are you happy with just the one that "worked" being written up in isolation?
Re:Effect size is significantly more important than statisti...
by harry8 on Thursday September 16, 2021 @08:41PM
(#28559696)
Especially for small effect size and suppressing what didn't work, this is one obvious way of many to perform p-hacking for publication acceptance.
Re:Effect size is significantly more important than statisti...
by hprotagonist on Wednesday September 15, 2021 @02:12PM
(#28542398)
i don’t want to know about each test that didn’t work as a separate publication, that’s for sure!
Re:Effect size is significantly more important than statisti...
by carlmr on Wednesday September 15, 2021 @03:37PM
(#28543412)
That's true, we would need a way to collect this data so it's easily seen as part of a whole.
E.g. if you search for eggs and cholesterol you should find all studies with their summarized results on whether eggs are ok or not for your cholesterol, grouped by researcher, so if somebody does 200 studies to find the one positive it's instantly visible.
Re:Effect size is significantly more important than statisti...
by PeterisP on Thursday September 16, 2021 @08:15AM
(#28550759)
You would read a meta-study that summarizes those tests - especially because they might potentially made by different labs, and the fact that one of them worked might be actual a real effect caused by some other difference in the experiment.
Re:Effect size is significantly more important than statisti...
by NavinF on Tuesday September 14, 2021 @11:31PM
(#28534936)
If someone really tested those hypotheses, let them publish. I doubt they'll get funding so it'll be on their own dime. In practice people do run experiments like that, but they only publish the 1/4 trials that is successful.
Re:Effect size is significantly more important than statisti...
by naasking on Tuesday September 14, 2021 @11:32PM
(#28534939)
Look to physics for how negative results should be published. There typically has to be reason to suspect some dosage range should work, in which case that sequence of studies you describe would be perfectly valid if it's within that range.
Re:Effect size is significantly more important than statisti...
by captain_price7 on Tuesday September 14, 2021 @10:40PM
(#28534582)
Why would someone want to game a negative result? Nobody ever becomes famous for saying my approach doesn't work. (As long as science is open, to make sure there is actually good work done by researchers before reaching this neg result.)
Re:Effect size is significantly more important than statisti...
by candiodari on Wednesday September 15, 2021 @05:46AM
(#28537133)
To have their name on a publication, which is a currency in the academic world.