News for hackers. Stuff that matters.
Re:Guide to Linear Regression (2015) by hackaflocka on Thursday January 21, 2016 @03:02PM (#10947884)
IMHO, a very confusing thing about Linear Regression is that there's widespread disagreement about whether the data are supposed to be Normally Distributed or not, and on how to measure said Normality.
Re:Guide to Linear Regression (2015) by stdbrouw on Thursday January 21, 2016 @03:45PM (#10948171)
You're right, there is lots of confusion in the topic and not all texts on regression get it right. Theoretically, linear regression depends only on a normally distributed outcome variable, the predictors don't have to be normally distributed. Practically speaking even that requirement can mostly be ignored: OLS is fairly insensitive to non-normal outcomes. Violations of the assumptions of regression usually don't affect the estimates much, they mainly affect the uncertainty around those parameter estimates -- the standard errors. This too is easily solved: use bootstrapping to calculate the standard errors.

In short: you can't just throw linear regression at anything and expect to get reasonable results, but it's pretty damn close.

Re:Guide to Linear Regression (2015) by hooloovoo_zoo on Thursday January 21, 2016 @07:00PM (#10949601)
Even that normality is not required; see, for instance, https://en.wikipedia.org/wiki/Gauss%E2%80%93Markov_theorem.
Re:Guide to Linear Regression (2015) by stdbrouw on Friday January 22, 2016 @04:20AM (#10951698)
Good catch. Outcome (or error, which boils down to the same thing) can't just be anything either, though, e.g. a common pattern is that the variance in an outcome increases when the predictor increases (e.g. a machine behaves more erratically when it spins faster) and that can mess with estimates.
Re:Guide to Linear Regression (2015) by fats_tromino on Thursday January 21, 2016 @03:15PM (#10947973)
Also some features of regression and hypothesis testing are actually fairly robust to non-normality, particularly with large sample sizes.
Re:Guide to Linear Regression (2015) by fats_tromino on Thursday January 21, 2016 @03:10PM (#10947955)
Look at normal qq-plot to measure normality