Re:Measuring the popularity of different vector databases
by freediver on Tuesday September 20, 2022 @02:56PM
(#32916251)
Another upvote for ScaNN. Used on Teclis.com search engine in production (replaced previously used Faiss, which was excellent). ScaNN is almost double the performance and with same 5 lines of code to implement locally.
At this point vector search is almost 'solved', the challenge remains the quality of vectors fed into the index. Producing accurate text/image/... vector representations (aka embeddings) is where lot more progress needs to be made.
Re:Measuring the popularity of different vector databases
by moab on Tuesday September 20, 2022 @03:36PM
(#32916709)
There are many "extra" features that one needs in practice which are patently not solved, starting with working in a dynamic setting.
Re:Measuring the popularity of different vector databases
by thirdtrigger on Tuesday September 20, 2022 @03:09PM
(#32916415)
> At this point vector search is almost 'solved'
Well, there is is still a lot to do. Going from memory to disk, etc. But I see your point.
What might be a bit confusing is the mix of vector search libraries with vector search engines. It's a bit like comparing an inverted index library with SOLR :)
But we (I.e., the wider vector search ecosystem) are working on this
We use ScaNN for a large scale/performant neural search. Otherwise this all feels bloated.