News for hackers. Stuff that matters.

A comment on Facebook releases 300-dimensional pretrained Fasttext vectors for 90 languages.

Re:Facebook releases 300-dimensional pretrained Fasttext vec... by minimaxir on Thursday March 02, 2017 @09:07AM (#13774043)
Played a bit with fastText months ago. The issue I had with it is that unlike CNNs/RNNs, the relative position of a word doesn't matter as much (only as a part of a context window during training the embeds), and so results can be worse depending on the case. However, for CPU-workloads, fastText is certainly faster, especially since subword information is also incorporated.

I have Python code to process text files into a fastText/friendly format so I may clean that up and see how these pre-trained embeds work.

(although, the English embeds are 10.36 GB; that might be a tough pill to swallow for training on machines with only 8 GB of RAM)

[ Parent | on HN ]
Re:Facebook releases 300-dimensional pretrained Fasttext vec... by exgrv on Thursday March 02, 2017 @10:20AM (#13774603)
Regarding the size of the word vectors files: the text files are sorted by frequency, so it is possible to easily load the top k words only.

We might also release smaller models in the future, for training on machines without large memory.

[ Parent | on HN ]
Re:Facebook releases 300-dimensional pretrained Fasttext vec... by morenoh149 on Friday March 03, 2017 @10:18AM (#13783387)
fwiw I have 32gb on my workstation and my personal laptop is maxed out at 16gb. Keeping within these thresholds may be useful to others.
[ Parent | on HN ]