News for hackers. Stuff that matters.
Re:When does peer review make no sense? by eanzenberg on Tuesday February 16, 2016 @02:36PM (#11112524)
When did p < 0.05 become acceptable? I remember in astronomy/astrophysics where 5sigma (3x10^-7 probability) was always the minimum required to publish.
Re:When does peer review make no sense? by Amorymeltzer on Tuesday February 16, 2016 @02:53PM (#11112657)
0.05 was first proposed by Fisher himself, which is 2 sigma if you look at a normal distribution (95% falls within 2 standard deviations[1]). Indeed, physics (generally particle physics, such as the Higgs Boson) goes for 5 or 6 sigma.

In biology, 0.05 is used because, well, it's hard otherwise. The cost, time, effort required for plants, animals, and even cells get exorbitant quickly, and are subject to many, many variations (animals generally have additional IACUC limits on procedures and numbers).

I could make arguments about how particles obey laws while biology is at least a few levels abstracted over that fundamental behavior, but xkcd has already done that. I personally feel that p<0.05 is absurd, and the entire biological field NEEDS to move to p<0.01. There's a movement toward that end, and it's about damn time.[3]

1: https://en.wikipedia.org/wiki/68%E2%80%9395%E2%80%9399.7_rul...

2: https://xkcd.com/435/ & https://xkcd.com/1052/ & https://xkcd.com/1520/ spring to mind

3: A lot of internet ink has been spilled on the topic, but http://www.nature.com/nmeth/journal/v12/n3/full/nmeth.3288.h... & http://www.pnas.org/content/110/48/19313.long & http://www.nature.com/news/scientific-method-statistical-err... are good places to start.

Re:When does peer review make no sense? by throwawayukcyb on Tuesday February 16, 2016 @02:48PM (#11112622)
social sciences and natural sciences have pretty much always had vastly different standards for what is acceptable. Getting 5 sigmas in many sorts of psychological studies would be pretty much impossible.

I very much doubt there was ever a time when experimental psychologists or sociologists held themselves to that standard.