Re:Learning Word Vectors for 157 Languages
by rspeer on Friday February 23, 2018 @05:32PM
(#16450200)
I argue that you need both domain-general and domain-specific data. Most domains you could apply NLP to don't have billions of words to train on, especially if the language of the data isn't English.
Pre-trained data is valuable, and you really aren't going to re-learn it all from your data, so why throw it out?
Re:Learning Word Vectors for 157 Languages
by laughingman2 on Friday February 23, 2018 @09:34PM
(#16451304)
Yes, recent work on transfer learning https://arxiv.org/abs/1801.06146 " Fine-tuned Language Models for Text Classification" suggests that training a lstm on a large corpora for language modeling, then using it as sentence encoder after tuning it in specific dataset gives a considerable perfomance.
This is covered in fast.ai 's new course 1.
Word vectors are fascinating representations. There is a huge amount of information and nuance captured in them. You can use them directly for topic retrieval (using annoy or another optimised vector index), or feed them into a classifier such as those in the Sklearn library. All types of neural nets: fully connected, recurrent and convolutional can be applied on word vectors.