News for hackers. Stuff that matters.
Re:Supercharge vector search with ColBERT rerank in PostgreSQL by nighthawk454 on Friday January 24, 2025 @03:01AM (#42811352)
Usually, we destructively compress (mean-pooling) both the query and the document, and then compare the two compressed forms.

With ColBERT, we compare first - at the full token level - for more detailed comparison. Then reduce the full set of comparisons to a single vector. Naturally this takes more memory and compute to do the more comparisons. The idea is it’s worth it because the more detailed comparisons lead to better results

tokens —> reduced vector —> comparison

Vs

tokens —> comparisons —> reduced vector

Re:Supercharge vector search with ColBERT rerank in PostgreSQL by kroolik on Friday January 24, 2025 @03:37AM (#42811503)
Why do you need the vector if you have already compared the query with the result candidate?
Re:Supercharge vector search with ColBERT rerank in PostgreSQL by nighthawk454 on Friday January 24, 2025 @04:38AM (#42811731)
Sorry, you’re right, it pools again to a single comparison scalar in the end
Re:Supercharge vector search with ColBERT rerank in PostgreSQL by lysecret on Friday January 24, 2025 @08:32AM (#42812844)
That’s actually a very good explanation thanks!