One issue is the kind of problem.

I remember getting 95%-ish accuracy with BoW and the SVM circa 2004 when it came to questions like "is this paper about astrophysics or organic chemistry?"

In that case you have a distinct vocabulary for different topics and it is hard to beat BoW.

Sentiment analysis, on the other hand, is where BoW goes to die since now "not good" means something very different than "good", and even simple heuristics like treating "not X" as a term that is different from "X" give limited gain because negation is expressed with constructions like "i don't believe that is good" and there is no k-word window that you reliably catch negation in since there isn't a limit on how complex sentences are.

There is also the question of "is the improvement between method A and method B worth it?" For instance the Netflix prize was much celebrated because some brilliant people busted their ass to go from 92% to 95% accuracy on movie recommendations. In the end the algorithm proved to be too complex for the value it created. (eg. Who would notice that they got 8 bad recommendations instead of 5 out of a hundred? An additional half a bad recommendation out of 10?)

The real "Netflix optimization problem" is how to spend as little on acquiring content as possible while motivating people to keep their subscriptions and that is something Netflix will keep closer to their chest and not promote a public competition on. (eg. if it were valuable why would they let competitors know about it?)

Good point, I should have written "for machine learning algorithms on problems that the industry is currently interested in".

It is important to understand "worked better". In practice, I would trade off using simpler methods for few points increase in precision/recall/your choice of metric.

You're assuming that they aren't trying: it's not too hard to try out bag-of-words and see what happens. But things like Attention and LSTMs are pretty good, but not without their costs.

I'm assuming the opposite: bag-of-words is the go-to baseline, but as another commenter pointed out, just separating instances based on vocabulary is not sufficient in today's sophisticated text classification problems.