We're building an AI data analyst. You can ask questions of your database and get answers immediately. We also auto generate entire dashboards based on common patterns (e.g. a "Sales Dashboard", "Marketing Dashboard", "Finance / Burn" etc.).
If you want to give it a try (there's a demo database embedded in the app), you can use it here: https://ui.definite.app/
Not that document databases don’t have their place, but…MongoDB is webscale and all that.
I ended up choosing Weaviate specifically because of the nice docs, but beyond that, time will tell.
https://weaviate.io/developers/weaviate/api/graphql/filters
https://weaviate.io/blog/hybrid-search-explained
I have a ChatGPT session where I have asked it to do a hybrid search using filtering, pg fts and vector search. Looks reasonable just need to test it and write it up somewhere.
Current concerns are the scaling and recall performance when looking at large amounts of records.
The author is looking at product quantization along with other ideas: https://github.com/pgvector/pgvector/issues/27
More details on product quantization: https://mccormickml.com/2017/10/13/product-quantizer-tutoria...
A nice repo that tracks the ANN relative performance of different indexes: https://mccormickml.com/2017/10/13/product-quantizer-tutoria...
Also shoutout to Weaviate because they have great docs, are open source and have very informative YouTube channel.