News for hackers. Stuff that matters.

A comment on Daniel Kahneman “I placed too much faith in underpowered studies”. Parent comment

Re:Daniel Kahneman “I placed too much faith in underpowered ... by nostrademons on Tuesday September 12, 2017 @02:44PM (#15230281)
You can if you admit the possibility of failure. Design an experiment that if it gets a significant result, that result will be solidly significant and counterintuitive. The flip side of that is that most of your results will not be significant, i.e. they will support the accepted scientific consensus. (And these should be published too - lack of interest in reproduced results is what triggered the reproduction crisis.)

It's much like with tech startups - you can design a startup such that it will either be worth billions or zero by tackling a problem that most people believe to be unsolvable but will affect lots of people, and then trying multiple counterintuitive ways of solving it. The vast majority of them will fail because that's what it means to be counterintuitive, but if it succeeds, you have a world-changing company.

Financial instruments as well - an option is a financial contract whose value will either be zero or a lot, depending on whether an unlikely-but-possible event occurs.

[ Parent | on HN ]
Re:Daniel Kahneman “I placed too much faith in underpowered ... by freshhawk on Tuesday September 12, 2017 @07:27PM (#15233983)
Sooner or later everyone will get off their collective asses and start registering studies before they are performed so that this is enforced and tracked. Hopefully.
[ Parent | on HN ]
Re:Daniel Kahneman “I placed too much faith in underpowered ... by tripzilch on Thursday September 14, 2017 @07:13PM (#15253258)
Funnily enough, some research into parapsychology (and similar)--being, the serious research that goes into stuff barely at the edge of pseudoscience, does exactly this. Preregistration is an important part to even begin having any experiments into "remote sensing" or "extrasensory perception" taken seriously by the community. The "serious" part of it, any way. There's a bunch of statisticians who love analysing this kind of thing, and there's parapsychology researchers that love drafting up experiments that challenge and skirt the edges of experimental science.

Not nearly enough of them of course.

But still, it's kind of funny how certain fields in pseudoscience actually drive progress in statistics and scientific methodology :)

[ Parent | on HN ]