Re:The Competitive Landscape for Machine Intelligence
by jonathankoren on Friday November 04, 2016 @03:12PM
(#12875106)
If you're looking at model feature weights, you're doing it wrong.
Most models aren't interpretable, and coefficients are highly unit and feature dependent. Discussions involving feature weights beyond "What if we reduce the feature space?" or "Did we implement this feature correctly?" often go bad, they almost always go bad when you're using it to "audit" the model. I have been in way too many discussions where someone suggested that the weights were wrong, simply because they thought something should "be more important".
One example is when you fit sparse high dimensional models to complex data in a real-time production system. The resulting models may have hundreds of millions to billions of features with non-zero weights, that constantly change as the underlying data changes. It's impossible to "hand-code" such a model from scratch by any reasonable size team in real-time. On the other hand, these hundreds of millions of rules can (and should) be exhaustively analyzed / audited by slicing and dicing both the model feature-weights, as well their performance on the data comprehensively. As an example, the "R" programming language typically creates useful human interpretable summaries for the models it generates.
For reference, I have been involved at Google in building such massive high dimensional models for properties like Youtube, and currently a founder of one the companies in the HBR report (LiftIgniter, YC W2014). Hopefully that doesn't make me too biased to respond.