News for hackers. Stuff that matters.
Re:Postgres full text search is good enough by llllllllllll on Monday September 29, 2014 @11:06AM (#8383196)
It can be done quickly and accurately. Check out Peter Norvig's implementation of Viterbi algorithm for text segmentation.

Segmentation is typically used for scripts of languages like Thai and Khmer that don't feature word boundaries. I don't know the ins and outs of German word compounding, but it should work for breaking apart compounds too given a fully conjugated/declined word list as input.

There are no comments on this comment yet.