Re:Haystack DB – 10x faster than FAISS with binary embedding...
by anewhnaccount2 on Monday April 29, 2024 @01:38AM
(#40194827)
The performance numbers do not show time performance versus accuracy perforamnce. Binary embeddings clearly have a time advantage, at the expense of accuracy. Reporting/benchmarking in this domain requires (at least) two dimensions.
[ Parent
| on HN ]
Re:Haystack DB – 10x faster than FAISS with binary embedding...
by carsonpoole on Monday April 29, 2024 @01:54AM
(#40194914)
I wholeheartedly agree :) I've mostly put this together this weekend, so I haven't had time to do everything yet, but that's definitely on the todo list.
I FT'd some siglip models (see here: https://huggingface.co/carsonpoole/binary-siglip-text) that should be amenable to binary quantization so I'm going to tomorrow get an inference server running with that and then hopefully do the typical MTEB benchmarks for embeddings
Re:Haystack DB – 10x faster than FAISS with binary embedding...
by DougBTX on Monday April 29, 2024 @06:18AM
(#40196545)
There are also FAISS binary indexes[0], so it'd be great to compare binary index vs binary index. Otherwise it seems a little misleading to say it is a FAISS vs not FAISS comparison, since really it would be a binary index vs not binary index comparison. I'm not too familiar with binary indexes, so if there's a significant difference between the types of binary index then it'd be great to explain what that is too.
[0] https://github.com/facebookresearch/faiss/wiki/Binary-indexe...