News for hackers. Stuff that matters.
Re:Postgres Full-Text Search: A search engine in a database by jka on Tuesday July 27, 2021 @04:02PM (#27976907)
"Faceted search"[1] (aka aggregates in Elasticsearch) tends to be a popular one, to provide user-facing content navigation.

That said, simonw has been on the case[2] demonstrating an implementation of that using Django and PostgreSQL.

[1] - https://en.wikipedia.org/wiki/Faceted_search

[2] - https://simonwillison.net/2017/Oct/5/django-postgresql-facet...

Re:Postgres Full-Text Search: A search engine in a database by craigds on Tuesday July 27, 2021 @06:10PM (#27978312)
This is the key one for us that makes Postgres a non-starter for FTS (We use postgres for everything else)

We begrudgingly use Solr instead (we started before ES was really a thing and haven't found a need to switch yet)

When you get more than about two different types of filters (e.g. types of filters could be 'tags', 'categories', 'geotags', 'media type', 'author' etc), the combinatorial explosion of Postgres queries required to provide facet counts gets unmanageable.

For example, when I do a query filtered by `?tag=abc&category=category1`, I need to do these queries:

  - `... where tag = 'abc' and category_id = 1` (the current results)

  - `count(*) ... where tag = 'def' and category_id = 1` (for each other tag present in the results)

  - `count(*) ... where tag = 'abc' and category_id = 2` (for each other category present in the results)

  - `count(*) ... where category_id = 1`

  - `count(*) ... where tag = 'abc'`

  - `count(*)`
There are certainly smarter ways to do this than lots of tiny queries, but all this complexity still ends up somewhere and isn't likely to be great for performance.

Whereas solr/elasticsearch have faceting built in and handle this with ease.

Re:Postgres Full-Text Search: A search engine in a database by radiospiel on Wednesday July 28, 2021 @06:44AM (#27982304)
You could run this in a single query `count(*) .. group by (tag, category_id)` and then post process the results.

counting by groups is still relatively expensive though in postgres; I built an implementation which uses estimate counting (via query planner) for larger buckets, and exact counting for only smaller buckets, but my use case allowed for some degree of inconsistency.

Re:Postgres Full-Text Search: A search engine in a database by kalev on Tuesday July 27, 2021 @11:15PM (#27979904)
This is exactly the issue I’m currently facing. We do a bunch of count queries to calculate facets and am looking for something that can do this out of the box. I’m glad I came to the same conclusion myself, either solr or elasticsearch might be the way to go. Starting this from scratch, which of the two would you recommend and why?
Re:Postgres Full-Text Search: A search engine in a database by Ueland on Wednesday July 28, 2021 @01:34AM (#27980541)
I myself is currently setting up Solr for a project as my experience with ES as a DevOps is not a happy one. Always nodes/indexes having some kind of problem. In the same time i have also worked with Solr for years before and never met any major issues. It just works and does the job well.
Re:Postgres Full-Text Search: A search engine in a database by craigds on Wednesday July 28, 2021 @01:35AM (#27980546)
Unfortunately I don't know enough about elasticsearch to provide a useful comparison. I like the way ES queries are structured JSON instead of obscure compact querystring parameters, but that's not a good reason to choose one over the other :)