Machine Learning Done Wrong

Re:Machine Learning Done Wrong by karthikv2k on Saturday February 28, 2015 @09:51PM (#9125850)

"2. Use plain linear models for non-linear interaction" It should be noted that Linear models are only linear in the model parameters, while the features can be transformed using non-linear functions. This trick makes linear models very powerful. Also if you have big data (in millions/billions) then you are better off with linear models, as SVM is very difficult to scale.

In my experience (all in big data), I rarely seen people use SVM, usual choices are logistic regressions and tree based models. In some finance and insurance industries you are restricted to use only interpretable models, which linear models are.

[ Parent | on HN ]

Re:Machine Learning Done Wrong by chengtao on Saturday February 28, 2015 @11:36PM (#9126042)

As you pointed out the transforming features is powerful, I believe that's the exact reason which makes SVM powerful. Though the way features can be combined with SVM is limited, the limitation makes SVM training fast in the dual space.

On the other hand, if you wanna compare logistic regression with SVM. While the detail is pretty tricky. One simplified view is to compare linear SVM which is essentially hinge loss with L2 regularization against logistic regression with L2 regularization which is essentially negative binomial log likelihood loss with L2 regularization. If you plot the loss functions, it's easy to see how they penalize negative & positive cases differently.

[ Parent | on HN ]