News for hackers. Stuff that matters.
Re:Understanding word vectors by ppod on Friday March 02, 2018 @12:27PM (#16503634)
Interesting. I had heard specifically that word vectors weren't a game-changer for document classification, because the averaging method didn't work well. But the difference between word2vec and FastText must be important. I think your baseline of Naive Bayes + TF-IDF would do a lot better if it was SVM and TF-IDF, but I didn't expect the big jump between word2vec and FastText.
Re:Understanding word vectors by wenc on Friday March 02, 2018 @12:51PM (#16503917)
> I had heard specifically that word vectors weren't a game-changer for document classification, because the averaging method didn't work well.

As with anything, your mileage may vary.

One aspect of FastText that definitely helped in my case was n-gram support (both word and character, tunable via command-line arguments). In my corpus, I have short fragments of sentences containing misspelled words, incorrect grammar etc. plus my test set has out-of-vocabulary words.

n-grams are more robust to these than Word2Vec which uses a static vocabulary.

Re:Understanding word vectors by swsieber on Friday March 02, 2018 @09:10PM (#16507434)
I thanked your top level comment, but this whole thread is great. I fond my self worth data that sounds like yours. I'm excited to try fast text.
Re:Understanding word vectors by sp332 on Friday March 02, 2018 @02:57PM (#16505106)
I think it's important to use the right "average" operation. word2vec puts words on the surface of a hypersphere, so "adding" vectors means turning around the sphere.