News for hackers. Stuff that matters.
Re:Postgres text search: balancing query time and relevancy by necovek on Friday October 15, 2021 @04:05AM (#28875503)
A nice exploration of Postgres trigram extension.

A few things I'd try if I was building a dedicated code search tool is to introduce custom per-language tokenizers for Postgres FTS that actually tokenize according to language rules (thus making "def" or "if" a stopword for Python, but also splitting "doSomethingCrazy" into ("do", "something", "crazy").

Then, I'd do two searches: one using such a FTS query first for more relevant results, and trigram search after. Combining results might be tricky, but not overly so imho (though the devil is in the details).

As for "limit interval to", you can approximate that by doing a LIMIT on unsorted results in a subselect (get the number with heuristics and adjust it as the data set grows), and then sort those by relevance: the result is effectively the same, except that you are using dataset size as the boundary instead of time.

Re:Postgres text search: balancing query time and relevancy by throwaway-kg8d5 on Friday October 15, 2021 @04:28AM (#28875598)
> you can approximate that by doing a LIMIT on unsorted results in a subselect

If you still have time left over, you can increase this LIMIT and run the query again (c.f. "iterative deepening") to get something like the desired time-limited query behaviour