Tabs, spaces and your salary – how is it really?

Re:Tabs, spaces and your salary – how is it really? by simonh on Tuesday June 20, 2017 @11:50AM (#14595536)

When you have large amounts of data, even if it's for a large number of people eventually youre going to find odd 'correlations' between unconnected variables. People who play edition X of Dungeons and Dragons are 30% less likely to eat chicken sandwitches than people who play edition Y. If you know a thousand things about everybody, the data is going to be riddled with bogus 'connections' like this.

In this case it's too small a sample for this finding to be significant, but increase the sample size and also increase the number of data points per person and you get similar problems. Determining significance is not straightforward.

[ Parent | on HN ]

Re:Tabs, spaces and your salary – how is it really? by var_explained on Tuesday June 20, 2017 @11:54AM (#14595587)

This is not the reason for the correlation. The p-value of the tabs/spaces connection is about 10^-13 (one in 10 trillion), and it can be found separately within each (large) country.

This does not mean the effect isn't confounded with some other factor, but it does mean it's not a multiple hypothesis testing issue.

You can explore the code and regression yourself! Take a look: https://github.com/dgrtwo/tabs-spaces-post

[ Parent | on HN ]

Re:Tabs, spaces and your salary – how is it really? by mrkgnao on Tuesday June 20, 2017 @12:06PM (#14595746)

http://www.tylervigen.com/spurious-correlations

[ Parent | on HN ]

Re:Tabs, spaces and your salary – how is it really? by sigstoat on Tuesday June 20, 2017 @11:56AM (#14595611)

yup, the multiple comparisons problem: https://en.wikipedia.org/wiki/Multiple_comparisons_problem

a common reason for a study to be total junk.

[ Parent | on HN ]