News for hackers. Stuff that matters.
Re:Dense Vector and Sparse Vector and Fulltext and Tensor Re... by small-turtle on Tuesday July 16, 2024 @09:57PM (#40981941)
How do you compare with other vector databases? some of them have already implemented both dense vector and sparse vector search.
Re:Dense Vector and Sparse Vector and Fulltext and Tensor Re... by yingfeng on Tuesday July 16, 2024 @10:23PM (#40982038)
There are some vector databases that already include both dense vector search and sparse vector search, such as qdrant. A hybrid search of these 2 does not solve many problems well, such as exact queries. Moreover, according to our experiments, as seen in the article, the performance of dense vector + sparse vector, improves only a little bit. In addition to these 2 way recall, infinity offers bm25 as well as colbert reranker, which can make the ranking quality of the hybrid search much better.