Re:Machine Learning Done Wrong
by kafkaesk on Sunday March 01, 2015 @07:37AM
(#9126838)
I have no professional experience with ML, so I might be missing something obvius here that's part of the industry paradigm.
But the article gives two points why you shouldn't use coefficient values to determine feature importance, which I think are only valid to some extent.
>a) changing the scale of the variable changes the absolute value of the coefficient
and
>(b) if features are multi-collinear, coefficients can shift from one feature to others.
Regarding a), well, that's what standardized coefficients are for.
b) is a bit trickier, but most regression models are based on the assumption of non-collinearity. This is of course a problem with real-world data, because you will quite often find some level of collinearity. That's when you (1) test for this issue and (2) look towards multilevel models.
But the article gives two points why you shouldn't use coefficient values to determine feature importance, which I think are only valid to some extent.
>a) changing the scale of the variable changes the absolute value of the coefficient
and
>(b) if features are multi-collinear, coefficients can shift from one feature to others.
Regarding a), well, that's what standardized coefficients are for.
b) is a bit trickier, but most regression models are based on the assumption of non-collinearity. This is of course a problem with real-world data, because you will quite often find some level of collinearity. That's when you (1) test for this issue and (2) look towards multilevel models.