Re:Do we think about vector dbs wrong?
by osigurdson on Tuesday September 05, 2023 @09:11AM
(#37391136)
An OpenAI embeddings vector is 1536 4 byte floats. 1 TiB is roughly 174K such embeddings vectors.
Re:Do we think about vector dbs wrong?
by brigadier132 on Tuesday September 05, 2023 @10:02AM
(#37391759)
Your math is wrong. 100k 32 bit vectors is 600mb
I think your point is right though. Searching through these is requires an index of some sort at any reasonable scale (not google scale).
Re:Do we think about vector dbs wrong?
by osigurdson on Tuesday September 05, 2023 @10:46AM
(#37392382)
Hmmm, yeah, I used 2^30 instead of 2^40. Should not comment before caffeinating.
Re:Do we think about vector dbs wrong?
by drittich on Tuesday September 05, 2023 @09:01AM
(#37391051)
And if you just have tens of thousands, straight up SQL can work pretty well, too.
If you have tens of thousands, a `for` loop over an array in memory is plenty fast.
I've worked on a project that searched through 50 million vectors with linear brute-force search, just with a sprinkle of AVX (and it used the brute-force search, because the curse of dimensionality killed all the smarter approaches).
Do you have more than a terabyte of embeddings? No? Do yourself a favor and use pg_vector with a HNSW index, it'll do just fine. Operationally, it is very hard to beat Postgres.