How to Think about Correlation?

Re:How to Think about Correlation? by rmrfstar on Wednesday December 09, 2020 @07:29AM (#25357843)

This is like... intro-to-intro statistics, summarized in the last bit of the article: "yes, corr is like a rescaled regression coefficient."

    y = x*b + e
    b = cov(x,y)/var(x) = corr(x,y) * (std(y)/std(x))

It's too bad the article did not mention omitted variable bias. The two principal sources of spurious correlation are (1) measurement error, (2) omitted variable bias. It is much easier to grok omitted variable bias in the regression context, then pull it back to correlation with the scaling rule.

[ Parent | on HN ]

Re:How to Think about Correlation? by monoideism on Wednesday December 09, 2020 @07:54AM (#25357990)

I last studied this stuff 20 years ago. What's the difference between ommitted variable and a confounding variable? Seems like a similar idea.

Is it that an ommitted variable isn't part of a causal chain of an included variable, and a confounder is?

[ Parent | on HN ]

Re:How to Think about Correlation? by rmrfstar on Wednesday December 09, 2020 @07:57AM (#25358012)

Confounding variable is a more confusing, less precise name for the same phenomenon.

Omitted variable bias tells it like it is. A bias in a regression coefficient that results from an incorrectly specified model.

[ Parent | on HN ]

Re:How to Think about Correlation? by monoideism on Wednesday December 09, 2020 @08:01AM (#25358038)

Thanks. Assuming they refer to the exact same phenomenon, I agree that ommitted variable bias is better terminology.

I wonder why Wikipedia has separate entries for them:

https://en.wikipedia.org/wiki/Omitted-variable_bias

https://en.wikipedia.org/wiki/Confounding

[ Parent | on HN ]

Re:How to Think about Correlation? by rmrfstar on Wednesday December 09, 2020 @08:11AM (#25358096)

Context.

The picture at the top of the confounding article gives it away. Those kinds of diagrams are common in "hierarchical Bayesian models" like LDA.

In the simple linear setting, they are the same thing. Teaching people the "confounding variable" concept in a general setting before teaching them about "omitted variable" in a linear setting is like teaching people about Riemannian manifolds before teaching them about vector spaces.

Correlation isn't a particularly useful concept outside of simple linear models.

[ Parent | on HN ]