News for hackers. Stuff that matters.
Re:P-Values are not Error Probabilities (2003) [pdf] by wodenokoto on Monday April 06, 2015 @10:40PM (#9331739)
> so they insist on following some strict rules, whether they make sense or not.

I took my bachelor in business and we were always told to make rational business decision. That is, informed decisions, based on data.

As for actual statistics, we were told 2 rules of thumbs: You need at least 30 people in a group before you can make meaningful statistics and it is significant if P < 0.005

Re:P-Values are not Error Probabilities (2003) [pdf] by RA_Fisher on Monday April 06, 2015 @11:56PM (#9331929)
Unfortunately that advice is hugely misleading. The reason is that the p-value essentially tells you how repeatable your analysis will be, but doesn't say anything about how beneficial making the decision might be. Second, there is a relationship between how large the difference between outcomes are and how many people you need. So if taking your conversion rate from 10 to 15% would be really helpful to your business, you might need say thousands of people to notice that difference statistically.
Re:P-Values are not Error Probabilities (2003) [pdf] by cgearhart on Tuesday April 07, 2015 @01:55PM (#9335798)
> The reason is that the p-value essentially tells you how repeatable your analysis will be

Your statement contradicts the introductory paragraph of the article:

> "...the outcomes of these tests are mistakenly believed to yield the following information:... the probability that an initial finding will replicate; ..."

The p-value is the likelihood that the observed effect is due only to chance, not a measure of the repeatability of a result.

Re:P-Values are not Error Probabilities (2003) [pdf] by RA_Fisher on Tuesday April 07, 2015 @05:27PM (#9337280)
Ah yes, you're right! I made a statement about likelihood!
Re:P-Values are not Error Probabilities (2003) [pdf] by wodenokoto on Tuesday April 07, 2015 @10:36PM (#9338511)
This was meant as an example of how crazily statistics is taught and why people end up just following strict rules.