But I get 1/5 - 1/10 hit ratio(successful/empty searches). That's not habit forming, memory forming for me.
Is there a core use case where I would get a good hit ratio ?
A similar concept was shown here recently: https://search.marginalia.nu
What was the largest index you've had on Faiss? That seems to affect whether people think of it as more than adequate or terribly inadequate.
If you do have a tiny index and want to try Google's version of vector search (as an alternative to Faiss), you can easily run ScaNN locally [1] (linked in the article, that's the underlying tech). On small scale I had better perf with ScaNN
[1] https://github.com/google-research/google-research/tree/mast...
I compared BERT[1], distilbert[2], mpnet[3] and minilm[4] in the past. But the results I got "out of the box" for semantic search were not better than using fastText, which is orders of magnitude faster. BERT and distilbert are 400x slower than fastText, minilm 300x, and mpnet 700x. At least if you are using a CPU-only machine. USE, xlmroberta and elmo were even worse (5,000 - 18,000x slower).
I also love how fast and easy it is to train your own fastText model.
[1]: distiluse-base-multilingual-cased-v2
[2]: multi-qa-MiniLM-L6-cos-v1
[3]: multi-qa-mpnet-base-cos-v1
[4]: multi-qa-MiniLM-L6-cos-v1
A concrete example is DPR which is a state of the art dense retriever model for wikipedia for question answering, when applying that model on MS Marco passage ranking it performs worse than plain BM25.