News for hackers. Stuff that matters.
Re:Do we think about vector dbs wrong? by rafaelero on Tuesday September 05, 2023 @09:48AM (#37391590)
> In 2023, there has been a shift towards acknowledging keyword search also has value and that a combination of vector + keyword search (aka hybrid search) operates in the sweet spot.

Do you have evidence that this hybrid approach is better than a vector search? That would be surprising.

Re:Do we think about vector dbs wrong? by gk1 on Tuesday September 05, 2023 @10:36AM (#37392266)
Yes, we published benchmarks here: https://www.pinecone.io/blog/hybrid-search/
Re:Do we think about vector dbs wrong? by lmeyerov on Tuesday September 05, 2023 @10:19AM (#37391989)
Imagine "day 2" search tasks like the user wanting to filter results for a specific time range

You could force the data team to go through hoops like multimodal embeddings, multiple indexes, post-filtering... or figure it out

Near-term, it makes sense to do one thing well, so many new vector DBs are really mutable vector indexes with lackluster search. But eventually, if you are not a narrowly-scoped kernel (faiss) but the main search platform needing $$$-level marketshare for growing payrolls, you're incentivized to figure search out beyond ANN, and your competitors are advertising that they are doing the same. So you add post-filtering, and then eventually, maybe even push-down/dynamic.

Source: We have been happily using on-the-fly GPU knn embeddings for years in pygraphistry, but when we added billion-scale vector indexing to power louie.ai RAG search for real-time news & database querying, we outgrow pure ann on our first user, and I'm thankful we did diligence on our DB selection

Re:Do we think about vector dbs wrong? by sroussey on Tuesday September 05, 2023 @10:49AM (#37392421)
And what did that due diligence reveal?
Re:Do we think about vector dbs wrong? by lmeyerov on Tuesday September 05, 2023 @02:00PM (#37395436)
Most prominent for a surprising amount of the vendors we looked at last year (https://gradientflow.com/the-vector-database-index/):

- We would have paid more for storing billion+ rows in-memory than our gov & enterprise customers were paying for a full product. Since then, ivfpq etc have gotten more popular to do responsive searches for bigger-than-memory datasets. If we needed small in-memory workloads, we could have done faiss+pandas+fastapi and moved on.

- Vendors pitched themselves as managed databases yet their users complain about them going down & losing data

- Many did not support basic search operators: time filtering, ...

- Few/none had real modern security, e.g., row-level multi-tenant ABAC

I haven't been tracking on a per-vendor basis as each publicizes making piecemeal progress on these. We found one that did #1-3 without failing us yet, and we're biting the bullet to implement #4. A scalable OSS answer to #4 might be the one that causes us to switch.

Re:Do we think about vector dbs wrong? by dmezzetti on Tuesday September 05, 2023 @09:53AM (#37391652)
In relation to txtai: https://neuml.hashnode.dev/benefits-of-hybrid-search

If you run a web search for "vector hybrid search" you'll come across plenty of articles for a number of vector databases discussing the topic.

Re:Do we think about vector dbs wrong? by softwaredoug on Tuesday September 05, 2023 @09:57AM (#37391693)
I'd actually want evidence for the opposite. Why vector retrieval for search? What's the evidence for that?

BM25 is often a baseline for improving upon. But I see improvements using lots of techniques, not just vector retrieval.

Re:Do we think about vector dbs wrong? by dmezzetti on Tuesday September 05, 2023 @10:13AM (#37391901)
It would be good to have a BM25 baseline entry on this leaderboard: https://huggingface.co/spaces/mteb/leaderboard

And I'm sure you're aware of the BEIR paper: https://arxiv.org/abs/2306.07471. Elastic references that in this blogpost: https://www.elastic.co/blog/improving-information-retrieval-...

I agree that BM25 retrieval + vector re-ranking can work. But vector search does bring results to the table that vanilla BM25 can't, even with a large retrieval window. So I do think there is a place for both with the usual "it depends on your data/requirements" caveat.

Re:Do we think about vector dbs wrong? by softwaredoug on Tuesday September 05, 2023 @10:17AM (#37391968)
The opposite is also true. BM25 prefers lexical matches and brings these candidates back that vector search often doesn’t.

I am not disagreeing vectors are useful, but I think benchmark based evidence is not the same as deploying a solution that must scale, be constantly updated, serve many use cases like filtering, search syntax, etc customers want.

And plus I think there’s a real danger Of herding to vector retrieval (even then one view of it) which cuts off exploration of diverse solutions.

Re:Do we think about vector dbs wrong? by dmezzetti on Tuesday September 05, 2023 @10:25AM (#37392100)
Hybrid retrieval can possibly be the best of both worlds.

I invested a lot of time with the last txtai release adding a minimal dependency Python-based BM25 component (https://neuml.hashnode.dev/building-an-efficient-sparse-keyw...). And keyword-only indexes are supported if one desires.

I'm with you 100% on not herding to any one way for any problem. I still remember the pre-2023 world where you weren't pressured to work LLMs into everything.