Every story and comment from the Hacker News API, in a layout borrowed from 2001.
A comment on Embeddings are a good starting point for the AI curious app developer. Parent comment
You also have various ways to separate the data for indexes/performance
- use metadata filtering first (eg: filter by customer ID prior to running a semantic search). This is fast in postgres since its a relational DB
- pgvector supports partial indexes - create one per customer based on a customer ID column
- use table partitions
- use Foreign Data Wrappers (more involved but scales horizontally)
There are no comments on this comment yet.
You also have various ways to separate the data for indexes/performance
- use metadata filtering first (eg: filter by customer ID prior to running a semantic search). This is fast in postgres since its a relational DB
- pgvector supports partial indexes - create one per customer based on a customer ID column
- use table partitions
- use Foreign Data Wrappers (more involved but scales horizontally)