Re:What it takes to build great machine learning products
by robrenaud on Monday April 16, 2012 @02:26PM
(#3848604)
Naive Bayes is a linear classifier, and it makes much stronger assumptions than other linear classifiers.
Re:What it takes to build great machine learning products
by irahul on Monday April 16, 2012 @03:55PM
(#3849047)
My bad.
Regarding stronger assumptions, is there anything other than independence(thus the name naive) that it assumes?
Re:What it takes to build great machine learning products
by tensor on Monday April 16, 2012 @04:18PM
(#3849182)
That's the main assumption that people care about. In contrast, logistic regression (aka maximum entropy) does not make this assumption. As a go-to first classifier I would suggest multi-class logistic regression with regularization.
I find data cleansing(if you are including feature selection) hard, and I consider it a refinement. If I am working on a classification problem, I start with naive bayes with a trivial feature generator(if words are feature, split on whitespace and discard some symbols), train it, and cross validate. Depending on the results of the cross validation on differently sized data-sets(say 100 tweets, 200, 500, 1000, 2000, 5000) I decide if I refine bayes further or I need to pick another algorithm.
I avoid SVM because I have a hard time figuring out the kernel and relation between data. I mostly don't use linear classifiers because the relation is very rarely linear.
Generally if the features are pseudo-independent(naive bayes assumes independent events but it might work fine even if the events aren't independent), naive bayes does the job. If not, it's time to refine the feature generator and selector.