News for hackers. Stuff that matters.
Re:Building a full-text search engine in 150 lines of Python... by gfxgirl on Thursday March 25, 2021 @09:04PM (#26587728)
Failed for me. I use Chinese and Japanese. There are no spaces to split on. I also search code where this also fails. I know it was meant to be simple and illustrate some points. I think the point it fails at is that this is actually a much harder problem in real life than a simple 150 line solution suggests.
Re:Building a full-text search engine in 150 lines of Python... by samcodes on Friday March 26, 2021 @01:46AM (#26589096)
if you use a library for Chinese/Japanese tokenization (which is harder because the lack of space), it seems like the rest of the code would work?