baur

CrateDB might be a good fit for full text and vector search (it’s SQL database but has dedicated clauses for FT and VS).

Curious how do you use PG for key/value and queue - do you use regular tables or some specific extensions?

I can imagine kv being a table with primary key on “key” and for queue a table with generated timestamp, indexed by this column and peek/add utilising that index.

Yep, CrateDB is not ACID but has eventual consistency at the row level.

I think for some use cases it's fine that some inserts are not visible in search results immediately. Also, once could tweak refresh_interval or use explicit refresh for "must be visible" inserts.

I'm wondering if CrateDB [https://github.com/crate/crate] could fit your use case without keeping 2 places to store and sync data.

It's a relational SQL database which aims for compatibility with PostgreSQL. Internally it uses Lucene as a storage and such can offer fulltext functionality which is exposed via MATCH.

Congrats on the launch!

Just wondering, do you have any plans to support CrateDB?

It supports SQL and understands PG protocol - perhaps supporting Postgres kinda already makes it close.

That's a nice idea. I guess it's better to stay in primitive type range (to avoid long arithmetics), so we can "compress" up to 15 items into a single prime_product making it less than 2^64 - for English words should be just fine, I don't expect many words with 16 and more letters.

UPD: Sorry, "up to 15" is a wrong phrasing. I checked once how "prime factorial" fits into primitive, and first 15 primes can fit into long. So it's possible to handle even more symbols if it's smth like "aaaaaaa"64 times because it would be just 2^64

Trie also can be used as a hashset/map, roughly it's like hash(obj).toString is a "word" for a trie and in the leaf we store the object.

It's https://en.wikipedia.org/wiki/Hash_array_mapped_trie which used in Scala's immutableMap https://dotty.epfl.ch/api/scala/collection/immutable/HashMap...

Nice implementation!

There is an option to get all suffixes without traversing subtree, but it comes with extra O(N) memory where N is combined length of all stored words - depending on case might be acceptable since memory for storing words itself is O(N) anyway. https://stackoverflow.com/a/29966616/2104560 (update 1 and update 3)