A Large-Scale Study of Programming Languages and Code Quality in GitHub (2014)

Re:A Large-Scale Study of Programming Languages and Code Qua... by j2kun on Sunday October 01, 2017 @06:14PM (#15380174)

They use... varying p-values? Can you do that? It almost looks like they're choosing p after the analysis is done...

[ Parent | on HN ]

Re:A Large-Scale Study of Programming Languages and Code Qua... by s3nnyy on Sunday October 01, 2017 @06:30PM (#15380241)

That is called p-hacking I guess?

[ Parent | on HN ]

Re:A Large-Scale Study of Programming Languages and Code Qua... by carbor1 on Sunday October 01, 2017 @08:28PM (#15380772)

No, not necessarily. It's standard practice to pick a p-value significance cut-off (0.05), but report the smallest such standard cut-off that any particular value meets. So "p < 0.001" is reported for values that meet that threshold. Anything over the cut-off is just not reported as significant.

[ Parent | on HN ]

Re:A Large-Scale Study of Programming Languages and Code Qua... by j2kun on Monday October 02, 2017 @01:26AM (#15381865)

That seems dishonest to me. They're saying that some results are more significant after the fact. Is there any mathematical justification for why this is OK?

[ Parent | on HN ]

Re:A Large-Scale Study of Programming Languages and Code Qua... by dx034 on Monday October 02, 2017 @07:30AM (#15383052)

As far as I know there is not. There is no such thing as "more significant", results either are significant or they're not.

[ Parent | on HN ]

Re:A Large-Scale Study of Programming Languages and Code Qua... by carbor1 on Sunday October 01, 2017 @08:29PM (#15380780)

That being said, the authors are doing a lot of comparisons here! They ought to be reporting Dunn-Bonferroni corrected p-values.

[ Parent | on HN ]