Re:How Google Code Search Worked: Regex Matching with a Trig...
by btown on Sunday December 02, 2018 @10:11AM
(#18582602)
Are there any good writeups on how to implement the trigram index side of this? For instance, the code alludes to the idea that you could store each trigram as a bitmap over documents, then the Boolean combinations become bitwise operations. I suppose you could keep the bitmaps in compressed (or even probabilistic sketch) form, then decompress only those required for a query, but is this the right intuition? Are there lower-level libraries or databases that do this kind of thing well, rather than rolling your own on a KV store or using something monstrous like ElasticSearch?
[ Parent
| on HN ]
Re:How Google Code Search Worked: Regex Matching with a Trig...
by straws on Sunday December 02, 2018 @03:31PM
(#18584388)
There's https://github.com/google/zoekt which provides a pretty straight-forward and memory-efficient implementation. Their design document goes a bit into what makes it fast:
Re:How Google Code Search Worked: Regex Matching with a Trig...
by hn_throwaway_99 on Sunday December 02, 2018 @11:49AM
(#18583059)
Note that Elasticsearch is built on top of Lucene (lucene.apache.org), which is a straight forward search indexer that creates a reverse index and document store, and gives pluggable low-level access to parts of the indexer path. Would be very easy to create a custom analyzer to create a trigram-based index (my guess is one already exists).
Re:How Google Code Search Worked: Regex Matching with a Trig...
by user5994461 on Sunday December 02, 2018 @11:24AM
(#18582934)
The project was released by the original author here. https://github.com/google/codesearch
There is no information on the subject besides the 3 blog posts and the proof of concept.
Re:How Google Code Search Worked: Regex Matching with a Trig...
by shabble on Sunday December 02, 2018 @10:25AM
(#18582667)
https://wiki.postgresql.org/images/6/6c/Index_support_for_re... is I think the slides for a conference talk on implementing within Postgres
Re:How Google Code Search Worked: Regex Matching with a Trig...
by JaggedNZ on Sunday December 02, 2018 @02:39PM
(#18584088)
Postgres FTS with trigrams (pg_trigram extension) works really well for many search applications. Most of the effort would be tuning the FTS dictionary.
Re:How Google Code Search Worked: Regex Matching with a Trig...
by dreamer_soul on Monday December 03, 2018 @04:20AM
(#18587405)
My issue with it is that a small term such as one word doesn't event match to a record which is weird, I'm using it with rails. Thinking of using elastic search to help with that