Effect size is significantly more important than statistical significance

Re:Effect size is significantly more important than statisti... by harry8 on Wednesday September 15, 2021 @03:28AM (#28536269)

    we tried using 0.10 mL, it didn't work
    we tried using 0.11 mL, it didn't work
    we tried using 0.13 mL, it didn't work
    we tried using 0.15 mL, it didn't work
    we tried using 0.17 mL, it didn't work
    we tried using 0.16 mL, it didn't work
    we tried using 0.18 mL, it didn't work
    we tried using 0.20 mL, it didn't work
    we tried using 0.14 mL, it didn't work
    we tried using 0.12 mL, it worked so we published

Do you want to know the ones that "didn't work" existed? Or are you happy with just the one that "worked" being written up in isolation?

[ Parent | on HN ]

Re:Effect size is significantly more important than statisti... by harry8 on Thursday September 16, 2021 @08:41PM (#28559696)

Especially for small effect size and suppressing what didn't work, this is one obvious way of many to perform p-hacking for publication acceptance.

https://en.wikipedia.org/wiki/Replication_crisis

[ Parent | on HN ]

Re:Effect size is significantly more important than statisti... by hprotagonist on Wednesday September 15, 2021 @02:12PM (#28542398)

i don’t want to know about each test that didn’t work as a separate publication, that’s for sure!

[ Parent | on HN ]

Re:Effect size is significantly more important than statisti... by carlmr on Wednesday September 15, 2021 @03:37PM (#28543412)

That's true, we would need a way to collect this data so it's easily seen as part of a whole.

E.g. if you search for eggs and cholesterol you should find all studies with their summarized results on whether eggs are ok or not for your cholesterol, grouped by researcher, so if somebody does 200 studies to find the one positive it's instantly visible.

[ Parent | on HN ]

Re:Effect size is significantly more important than statisti... by PeterisP on Thursday September 16, 2021 @08:15AM (#28550759)

You would read a meta-study that summarizes those tests - especially because they might potentially made by different labs, and the fact that one of them worked might be actual a real effect caused by some other difference in the experiment.

[ Parent | on HN ]