Linear regression by hand

Re:Linear regression by hand by Rainymood on Saturday January 20, 2018 @12:23PM (#16194201)

Can someone explain to me why this has (so many) upvotes? This is like elementary undergraduate econ stats and kind of trivial?

There's very little content either, it's literally a reformulation of the formula, no interesting graphs or geometric interpretation. What I expected from a title like "Linear Regression By Hand" was the minimization of some quadratic error function, by hand (i.e. using pencil and paper).

[ Parent | on HN ]

Re:Linear regression by hand by triplesec on Saturday January 20, 2018 @05:11PM (#16195696)

My suspicion, for the Eeyores round here, is that this has upvotes because people like to bookmark a reminder of references to remind you of the maths behind what most of us do automagically nowadays, for when we might need it. And often it's the comments that provide even better and advance sources, as is the case here. And indeed there are some programmers new to this kind of thing.

[ Parent | on HN ]

Re:Linear regression by hand by VHRanger on Saturday January 20, 2018 @02:06PM (#16194824)

I think a lot of people here didn't take econometrics courses.

If you see all regression problems under the foil of maximum likelihood estimation, you might not know that ordinary least squares regression has a closed form solution

[ Parent | on HN ]

Re:Linear regression by hand by theophrastus on Saturday January 20, 2018 @01:15PM (#16194527)

One issue that I nearly always find missing in intro discussions about linear regression is the near universal assumption of no error in the abcissal/"x" values. And while this is true-ish for time series data, (we know for certain which day we collected the data on - yet the same hour every day?), I'd be rich if I had a nickel for every time I saw standard linear regression done when the "x" had significant (and known) error. In which case you're biasing yourself unless you use some sort of 2d regression, like Deming.[1]

[1] https://en.wikipedia.org/wiki/Deming_regression

[ Parent | on HN ]

Re:Linear regression by hand by VHRanger on Saturday January 20, 2018 @02:28PM (#16194948)

Regression with measurement error is usually treated in much higher level statistics/econometrics classes.

If you're interested in this you can read more in Mostly Harmless Econometrics [1] about adressing this with IV methods

[1] http://www.development.wne.uw.edu.pl/uploads/Main/recrut_eco...

[ Parent | on HN ]

Re:Linear regression by hand by pacbard on Saturday January 20, 2018 @02:41PM (#16195021)

To build on this a little bit more, there are also generalized linear models that allow to specify the reliability (i.e., error level) of a variable.

Regarding, 2SLS models, I find them more useful to account for endogeneity in the model rather than measurement error. After all, measurement error is usually unobserved (otherwise you would just take it out). 2SLS “just” reweigh the point estimates by identifying the good variation in the instrumented variable using the instrument (for example, using draft lottery results to account for the endogenous choice to attend college).

[ Parent | on HN ]

Re:Linear regression by hand by thess24 on Saturday January 20, 2018 @02:43PM (#16195032)

I agree, it seems like no help just to list a formula to memorize. If someone knows enough linear algebra to understand what the formula represents, they can do the derivation. This link [1] is a good one if anyone is interested.

[1] https://eli.thegreenplace.net/2014/derivation-of-the-normal-...

[ Parent | on HN ]

Re:Linear regression by hand by abecedarius on Saturday January 20, 2018 @05:42PM (#16195836)

What I expected "by hand" to mean was something like a handmade analog computer. E.g. print your scatterplot, tape a penny over each data point, push a tack through the origin and let the printout swing around it under gravity until it comes to rest (while keeping it flat -- I guess I should've first stuck the printout onto some cardboard). Is there some generalization of this idea that lets the intercept vary too?

[ Parent | on HN ]

Re:Linear regression by hand by luckyt on Saturday January 20, 2018 @03:16PM (#16195188)

Well, unless you're doing original research, everything you write will be trivial to somebody.

[ Parent | on HN ]

Re:Linear regression by hand by mr_toad on Saturday January 20, 2018 @08:54PM (#16196479)

There aren’t that many good articles on basic statistics that are freely available on the net. Textbooks are expensive, and most of the free material is of dubious quality. Wikipedia is particularly terrible on Statistics.

[ Parent | on HN ]

Re:Linear regression by hand by bluusteel on Saturday January 20, 2018 @02:17PM (#16194897)

It seems like blog posts on simple statistical methods like this one land on the front page of HNews a lot more than one would expect.

[ Parent | on HN ]

Re:Linear regression by hand by thaumaturgy on Saturday January 20, 2018 @03:45PM (#16195322)

https://www.xkcd.com/1053/

I almost certainly know something I'd consider "trivial" that you haven't encountered yet. I try to be really excited when that happens.

[ Parent | on HN ]

Re:Linear regression by hand by baking on Saturday January 20, 2018 @04:22PM (#16195489)

I think that perhaps the issue is that machine learning courses skip over the fact that there is a trivial closed form solution to 99.9% of all real-world machine learning problems.

[ Parent | on HN ]