News for hackers. Stuff that matters.
Re:Vald: A Highly Scalable Distributed Vector Search Engine by hintymad on Friday April 09, 2021 @01:10PM (#26753138)
thanks. The challenge with post-filtering is that the K in TopK becomes unpredictable if I'd like to get a fixed page size. I may end up issuing multiple queries with progressively large Ks to get a single page of results.
Re:Vald: A Highly Scalable Distributed Vector Search Engine by etiennedi on Sunday April 11, 2021 @05:55AM (#26769029)
Exactly. See my reply on this comment thread comment thread to see how Weaviate solves the filter-issue by using an inverted index to produce a whitelist of IDs which is then passed to the vector index where non matching IDs are simply skipped.