News for hackers. Stuff that matters.
Re:Go-Freeling: Natural Language Processing in Go by mschoch on Sunday June 07, 2015 @12:18PM (#9675250)
If you just want to segment larger blocks of text into tokens you can try the segment library (it implements the word boundary portion of unicode annex 29):

https://github.com/blevesearch/segment

If you need more manipulation of tokens after segmentation/tokenization, you could look at the analysis sub-package of bleve. Its intended to be able to be used indepenently of the rest of the library.

https://github.com/blevesearch/bleve

There are no comments on this comment yet.