News for hackers. Stuff that matters.
Re:Linear Regression by em500 on Friday June 19, 2020 @12:58PM (#23576342)
Deming regressions have some serious drawbacks compared to OLS, most importantly that it's not scale/unit-invariant: if you rescale your (e.g. use house area in square metres instead of square feet to predict house prices) you get different results.

This may be less of an issue for the machine leaner who's only interested in point predictions, but it's a serious concern for us old fashioned statisticians who are more interested in inference.

Re:Linear Regression by pps43 on Friday June 19, 2020 @03:56PM (#23578301)
I would not call it a drawback. In regular regression you make an assumption that x is known exactly, and all errors are in y. In Deming regression you make another assumption, about the ratio of their variances (δ). If you change units, you need to change δ as well, and then there's no difference.
Re:Linear Regression by autokad on Friday June 19, 2020 @01:28PM (#23576771)
i wouldn't dare to say its a 'serious drawback'. as soon as you add ridge to regression (something very common for statisticians to do), it's also no longer scale/unit-invariant.

Edit: this is also not just true for ridge, but lasso as well

Re:Linear Regression by em500 on Friday June 19, 2020 @02:07PM (#23577220)
Personally I consider the lack of scale-invariance one of the main drawbacks of most common regularizers too. Again, not a big deal if all you're after are y-hat, a bit more concerning if you're interested in beta-hat.
Re:Linear Regression by autokad on Friday June 19, 2020 @05:48PM (#23579189)
i try not to stick too religiously to coefficient interpretation unless its a very simple and known problem. one missing variable could change the coefficients sign.
Re:Linear Regression by zwaps on Friday June 19, 2020 @08:06PM (#23580233)
Depends on whether the missing variable is correlated with the beta hats you care about ;-)