News for hackers. Stuff that matters.

A comment on Pg_vectorize: The simplest way to do vector search and RAG on Postgres.

Re:Pg_vectorize: The simplest way to do vector search and RA... by valstu on Wednesday March 06, 2024 @05:31AM (#39614434)
I assume you need to split the data to suitable sized database rows matching your model max length? Or does it do some chunking magic automatically?
[ Parent | on HN ]
Re:Pg_vectorize: The simplest way to do vector search and RA... by chuckhend on Wednesday March 06, 2024 @09:42AM (#39616425)
There is no chunking built into the postgres extension yet, but we are working on it.

It does check the context length of the request against the limits of the chat model before sending the request, and optionally allows you to auto-trim the least relevant documents out of the request so that it fits the model's context window. IMO its worth spending time getting chunks prepared, sized, tuned for your use case though. There are some good conversations above discussing methods around this, such as using a summarization model to create the chunks.

[ Parent | on HN ]