News for hackers. Stuff that matters.
Re:Haystack DB – 10x faster than FAISS with binary embedding... by gcr on Monday April 29, 2024 @12:46AM (#40194540)
Do your vectors change a lot?

What have you tried so far? As a strong baseline, storing 10e6 512-bit vectors in memory should take about 640MiB of RAM. A binary Hamming distance between a query and a gallery is [popcnt(query xor elt) for elt in gallery], which should be plenty fast implemented in vectorized SSE instructions. I’d be shocked if this took more than 10ms per query per thread on modern hardware.

Re:Haystack DB – 10x faster than FAISS with binary embedding... by Loic on Monday April 29, 2024 @03:55AM (#40195710)
You are spot on, 10ms internal time for 100000 molecules, 60ms with the "stack on top": https://www.chemeo.com/similar?smiles=CCCCO It is using a manually coded, most likely not that efficient, with popcnt.
Re:Haystack DB – 10x faster than FAISS with binary embedding... by fbdab103 on Monday April 29, 2024 @08:12PM (#40205820)
These are not bit vectors, but float64. I could probably convert to float32 to cut down the ram. This is a one time job, so no changes.

My initial attempt has been using the scikit-learn cosine similarity function[0] which lets you take a query matrix against a target one. The actual calculation is pretty fast and seems quite optimized (at least, I can see all of my cores max out when it does the calculation). Then I iterate across the results matrix, sorting for best matches, and record best seen to date.

[0] https://scikit-learn.org/stable/modules/generated/sklearn.me...