News for hackers. Stuff that matters.
Re:SpaCy v1.0: Deep Learning with custom pipelines and Keras by nl on Thursday October 20, 2016 @05:41AM (#12750893)
I use Spacy quite a lot for work and pleasure. I also regularly use (deep breath) NLTK/CoreNLP/OpenNLP/Rosette/OpenIE, and I've written bits and pieces in the area, so I have some perspective.

When I want to do something, I always use Spacy first. It's the 95% solution - it does 95% of the things you are ever likely to do. The only downside is that I haven't worked out how to avoid the slow startup time.

NLTK has more stuff. Some of it is crap, but some is useful. For example, if you want WordNet then NTLK is an easy way to get it.

CoreNLP gives you high quality POS tagging and named entity extraction. It's a Java library (but can run as a REST service) so that is good if you are on the JVM.

OpenNLP.. has an Apache license.

Rosette is expensive, and tuned entirely differently to any other text processing library. Has great multi lingual support.

OpenIE is by far the best open information extraction toolkit.

Obviously this misses the 1000 pound gorilla: Syntax/Parsey McParseface. This isn't exactly trivial to run yet outside of Docker. Slowly getting my stuff together on this.

Re:SpaCy v1.0: Deep Learning with custom pipelines and Keras by syllogism on Thursday October 20, 2016 @06:46AM (#12751185)
This was valuable, thanks. The user survey shows most people do use a variety of NLP libraries as well as spaCy.

I'd appreciate a pull request with WordNet support for spaCy. I've thought about it at various points, and I already use WordNet data in spaCy's lemmatizer.

Support for more languages will come. The plan is to have a model store, so that you can buy data packs for additional languages.

Accuracy will improve when we translate over the bi-LSTM models.

In short: we'll keep chipping away at the remaining 5%. The library's only about 1 year old, and I haven't yet managed to do a good job of helping other developers contribute. There's plenty more to come.

Re:SpaCy v1.0: Deep Learning with custom pipelines and Keras by dirtyaura on Thursday October 20, 2016 @05:43AM (#12750898)
May I ask what kind of work you do and what kind of NLP projects on your free time?
Re:SpaCy v1.0: Deep Learning with custom pipelines and Keras by nl on Thursday October 20, 2016 @05:58AM (#12750952)
Work is running a "data science" R&D program across a few universities and an engineering team.

Fun is lots of stuff. I have a question-answering thing which I'm slowly making progress on. I've written various (old) open source text things which still seem to be used. I do Kaggle. I wrote a contextual advertising server for a cycling website I ran (and ended up licensing some of the tech). I have a vague home research project trying to combine unstructured knowledge with structured knowledge graphs. Probably other stuff I've forgotten about.

Re:SpaCy v1.0: Deep Learning with custom pipelines and Keras by dirtyaura on Saturday October 22, 2016 @03:36AM (#12767675)
Thanks! I've been playing with Spacy lately. I've a product idea that requires NLP and I'm unfamiliar with NLP research and techniques. The idea itself is rather complex to implement, so I'd like to have a couple of interesting toy/side projects to hone my skills first. I hadn't thought Kaggle, that could provide a few good projects for understanding NLP tech better.