News for hackers. Stuff that matters.

A comment on A Large-Scale Study of Programming Languages and Code Quality in GitHub (2014). Parent comment

Re:A Large-Scale Study of Programming Languages and Code Qua... by carbor1 on Sunday October 01, 2017 @08:28PM (#15380772)
No, not necessarily. It's standard practice to pick a p-value significance cut-off (0.05), but report the smallest such standard cut-off that any particular value meets. So "p < 0.001" is reported for values that meet that threshold. Anything over the cut-off is just not reported as significant.
[ Parent | on HN ]
Re:A Large-Scale Study of Programming Languages and Code Qua... by j2kun on Monday October 02, 2017 @01:26AM (#15381865)
That seems dishonest to me. They're saying that some results are more significant after the fact. Is there any mathematical justification for why this is OK?
[ Parent | on HN ]
Re:A Large-Scale Study of Programming Languages and Code Qua... by dx034 on Monday October 02, 2017 @07:30AM (#15383052)
As far as I know there is not. There is no such thing as "more significant", results either are significant or they're not.
[ Parent | on HN ]
Re:A Large-Scale Study of Programming Languages and Code Qua... by carbor1 on Sunday October 01, 2017 @08:29PM (#15380780)
That being said, the authors are doing a lot of comparisons here! They ought to be reporting Dunn-Bonferroni corrected p-values.
[ Parent | on HN ]