Re:SpaCy v1.0: Deep Learning with custom pipelines and Keras
by butterm on Thursday October 20, 2016 @12:44PM
(#12753911)
Yup, I know about that but Displacy is just so much more beautiful.
Also, while NLTK's basic Twitter tokenizer is okay, I find that ARK's tokenizer [0] is much better. Similarly, for POS tagging of tweets, I am using the GATE POS tagger [1]. They have a Stanford model and I can hook it up with NLTK using the StanfordTagger class. In fact, this is the kind of integration that I am missing in Spacy.
[0] https://github.com/myleott/ark-twokenize-py [1] https://gate.ac.uk/wiki/twitter-postagger.html
So I do a lot of work with Twitter data - to the point where I have (many) custom Word2Vec models for Twitter data.
Tweeboparser is good, and the NLTK has a basic Twitter tokenizer built in. But I still often end up using Space for stuff anyway. For example, I was building a custom distance metric to explore Tweet clusters, and the Spacy word vectors were fine to get that working.
It's true that I dropped down to using Gensim or Spark's Word2Vec model for some more complex models though.