News for hackers. Stuff that matters.
Re:Do we think about vector dbs wrong? by pradn on Tuesday September 05, 2023 @09:15AM (#37391186)
The elephant in the room is the Curse of Dimensionality. As the number of vector dimensions increase, "nearest neighbor" search behaves in an surprising manner. In a high-dimensional cube randomly sampled vectors are mostly near the edge of the cube, not at all what you expect based on two or three dimensions. There's other hitches, too.

When using vector search, you're making two leaps - that your document can really be summarized in a vector (keeping its important distinguishing semantic features), and that it's possible to retrieve similar documents using high-dimensional vectors. Both of these are lossy and have all sorts of caveats.

What would be great is if these new vector DB companies show examples where the top-ten "search engine results page" based on semantic embedding / vector search is clearly better than a conventionally-tuned, keyword-based system like ElasticSearch. I haven't see that, but I have seen tons of numbers about precision/recall/etc. What's the point of taking 10ns to retrieve good vector results if its unclear if they're solving a real problem?

https://en.wikipedia.org/wiki/Curse_of_dimensionality

Re:Do we think about vector dbs wrong? by WinLychee on Tuesday September 05, 2023 @09:29AM (#37391369)
These vectors are lower-dimensional than traditional vectors though, aren't they? Vector embeddings are in the hundreds to low thousands range of dimensions (roughly between 128-1024), whereas TF-IDF has the same dimension as your vocabulary. It's also not just about being flat-out better, but about increasing the recall of queries, as you're grabbing content that doesn't contain the keywords directly, but is still relevant. You are also free to mix the two approaches together in one result set, which gives the best of both.
Re:Do we think about vector dbs wrong? by mk67 on Tuesday September 05, 2023 @10:31AM (#37392203)
Nobody used TF-IDF for vector lookups without applying a PCA first though.
Re:Do we think about vector dbs wrong? by pradn on Tuesday September 05, 2023 @10:45AM (#37392372)
The problems with dimensionality certainly show up even with 256 dimensions. PCA-ing down to a few hundred dimensions is still a problem, and then you have to deal with PCA lossiness too!
Re:Do we think about vector dbs wrong? by snowstormsun on Tuesday September 05, 2023 @02:47PM (#37396022)
Thanks, interesting take!
Re:Do we think about vector dbs wrong? by barefeg on Tuesday September 05, 2023 @03:09PM (#37396330)
You can try the 3 methods: keyword, knn, and hybrid on a collection in the AI domain here https://search.zeta-alpha.com/. YMMV sometimes knn has more interesting results, sometimes keyword is closer to what you wanted. Hybrid is sometimes also a good middle ground. But it fails when one of the retrievers has very bad results (because it will interleave them)