Re:Show HN: Full text search Project Gutenberg (60m paragraphs)
by StavrosK on Sunday January 24, 2021 @05:52AM
(#25891000)
I'm curious if you ever tried MeiliSearch for this, I tried it recently for something unrelated and had a very good experience with it. Since you have the corpus already, it might be worth trying and seeing if it speeds search up? I'd be interested in the results either way.
Re:Show HN: Full text search Project Gutenberg (60m paragraphs)
by gutensearch on Sunday January 24, 2021 @06:03AM
(#25891052)
Thank you for the suggestion! The demo looks great and I'm curious to see how they automatically handle language. It would be nice to add support for Chinese which is the great absent from this attempt, even at the cost of several other languages which in any case have few books transcribed to text in Gutenberg.
I will probably write a blog post once I've tried a few of the approaches suggested in this thread.
The current search box is a wrapper around Postgres phraseto_tsquery [1] whilst the Discovery tab uses plainto_tsquery, so you could play with either as an ersatz for some of these features for now, although special characters might get stripped or parsed incorrectly.
Do you know where the people you are talking about hang out online (for example, subreddits)? I'd love to get in touch with them once the features are built and for more general feedback.
[1] https://www.postgresql.org/docs/12/textsearch-controls.html