Introduction to Vector Similarity Search

Re:Introduction to Vector Similarity Search by Xenoamorphous on Wednesday July 12, 2023 @12:59AM (#36690628)

This is what we use: BERT sentence transformers to generate the embeddings (we used Universal Sentence Encoder before that and it was good too), and ElasticSearch for storage, which has a dense vector data type. It also has a cosineSimilarity function to run searches.

[ Parent | on HN ]

Re:Introduction to Vector Similarity Search by fzliu on Wednesday July 12, 2023 @01:14AM (#36690687)

I would not use Elastic for vector search due to its architectural limitations and poor performance when conducting vector search. https://zilliz.com/benchmark

[ Parent | on HN ]

Re:Introduction to Vector Similarity Search by Xenoamorphous on Wednesday July 12, 2023 @02:08AM (#36690970)

I should’ve said that we were already using it for actual search where embeddings similarity is just one component of the overall score. For pure vector stuff a dedicated solution will be faster.

[ Parent | on HN ]

Re:Introduction to Vector Similarity Search by dunefox on Wednesday July 12, 2023 @03:37AM (#36691478)

What about Solr?

[ Parent | on HN ]