Re:What Kagglers Are Using for Text Classification
by PaulHoule on Thursday December 27, 2018 @11:34AM
(#18770486)
One issue is the kind of problem.
I remember getting 95%-ish accuracy with BoW and the SVM circa 2004 when it came to questions like "is this paper about astrophysics or organic chemistry?"
In that case you have a distinct vocabulary for different topics and it is hard to beat BoW.
Sentiment analysis, on the other hand, is where BoW goes to die since now "not good" means something very different than "good", and even simple heuristics like treating "not X" as a term that is different from "X" give limited gain because negation is expressed with constructions like "i don't believe that is good" and there is no k-word window that you reliably catch negation in since there isn't a limit on how complex sentences are.
There is also the question of "is the improvement between method A and method B worth it?" For instance the Netflix prize was much celebrated because some brilliant people busted their ass to go from 92% to 95% accuracy on movie recommendations. In the end the algorithm proved to be too complex for the value it created. (eg. Who would notice that they got 8 bad recommendations instead of 5 out of a hundred? An additional half a bad recommendation out of 10?)
The real "Netflix optimization problem" is how to spend as little on acquiring content as possible while motivating people to keep their subscriptions and that is something Netflix will keep closer to their chest and not promote a public competition on. (eg. if it were valuable why would they let competitors know about it?)
I remember getting 95%-ish accuracy with BoW and the SVM circa 2004 when it came to questions like "is this paper about astrophysics or organic chemistry?"
In that case you have a distinct vocabulary for different topics and it is hard to beat BoW.
Sentiment analysis, on the other hand, is where BoW goes to die since now "not good" means something very different than "good", and even simple heuristics like treating "not X" as a term that is different from "X" give limited gain because negation is expressed with constructions like "i don't believe that is good" and there is no k-word window that you reliably catch negation in since there isn't a limit on how complex sentences are.
There is also the question of "is the improvement between method A and method B worth it?" For instance the Netflix prize was much celebrated because some brilliant people busted their ass to go from 92% to 95% accuracy on movie recommendations. In the end the algorithm proved to be too complex for the value it created. (eg. Who would notice that they got 8 bad recommendations instead of 5 out of a hundred? An additional half a bad recommendation out of 10?)
The real "Netflix optimization problem" is how to spend as little on acquiring content as possible while motivating people to keep their subscriptions and that is something Netflix will keep closer to their chest and not promote a public competition on. (eg. if it were valuable why would they let competitors know about it?)