generall

In this scenario posting list have to share all the same attributes as each element inside the list.

In IVF you can start checking conditions only in the final bucket. There are no guarantees if the bucket has any acceptable value, and there are no procedures to find the bucket which has acceptable values before scanning it.

IVF, unfortunately, is barely compatible with filtered search. It have to rely on post-filtering and retrieve more and more candidates if the result set is not big enough. If the query is in some form correlated with the filter, this approach quickly degrades to brute-force.

Surprised that the article doesn't mention filtering use-case at all.

https://qdrant.tech/articles/dedicated-service/ - we have some arguments on this

This article contains a lot of inaccuracies.

Based on your statements, like

> Qdrant stores both the vectors and the metadata in a sqlite database.

It looks like you have benchmarked local mode of qdrant. It doesn't even use vector indexes and is not designed for any kind of production usage.

For anyone reading this article, I urge you to do your own benchmarks and not rely on claims that do not have open source code attached to them to replicate the results

With a few optimization tricks, TL;DR: - ONNX inference in Rust - Embeddings cache & lookup - Parallel & Batch requests - hybrid search with full-text filtering + vector re-scoring

I would say that text text and vector search are orthogonal. Some scenarios are better with one, others with combination. But fitting vector search into the interface designed for text is limiting vector search potential

Benchmark is open source - https://github.com/qdrant/vector-db-benchmark You are welcome to fork it and make your own measurements, if you suspect something.

Benchmarks like https://big-ann-benchmarks.com/index.html#organizers are good for comparing algorithms, but not engines. They are focused on a single use scenario and do not cover variety of possible applications. Like, for example, how filtering affect the performance.

> Please expand on how you are defining full text search distinctly from sparse search to continue the discussion

In addition to the indexing algorithm, there is the tokenizer, which depends on the language, lemmatizer, synonyms, stop-words, and so on and so forth. In addition, the ranking function itself may be quite different and based on different rules. See how Meilisearch does it. Reducing full-text search to just a reverse index is a misconception

> Your criticism of the Weaviate ANN benchmarks isn't relevant to our discussion on Hybrid Search.

It is very much relevant, as I mentioned, in parallel processes

total_latency = max(BM25_latency, Vector_search_latency) + merge overhead

and my claim is that in specialized tools both BM25_latency and Vector_search_latency will be better than what the multi-tool system can provide.

> I have linked this to show that Weaviate has produced comparative benchmarks which was your original claim.

I don't see any comparisons in your benchmarks here - https://weaviate.io/developers/weaviate/benchmarks/ann

You just benchmarked yourself, that is not interesting and not helping.

> I also agree that it would be interesting to run ANN recall tests on several hardware configurations.

That is not the point. In our benchmark we run all engines on exactly the same machine to make it fair. Sometimes same configuration in different regions already gives very different performance on some cloud providers.

> There are a few arguments why adding sparse search doesn't require too much extra specialization

Full-text search != sparse search, that's a naive oversimplification. Btw, sparse search is on Qdrant roadmap, so we should be able to compare it's performance on benchmarks.

> Cross Encoder inference generally doesn't happen in the database itself thus it makes sense to use modules to process the additional ranking logic

That statement makes your argument that `A combined system have better end-to-end latency.` invalid

> Such benchmarks exist as in trengrj's initial response and we are working on them as well.

link or it didn't happen.

In your current benchmarks you advertise everywhere, you're just throwing in disproportionately powerful and expensive hardware. Even a full-scan can give good results under those conditions

Proposed architecture doesn't limit you to use self-hosted transformers only, you can use OpenAI just as easily. And you don't need a to install a "module" for that

The latency of the combination of parallel systems is equal to the slowest component. And obviously, specialized tools will be faster than a component of a multi-tool system cause while dedicated engines can invest in optimizing specific functionality, multi-tool engines are stuck in the integration hell.

A solution assembled for a specific task from highly specialized components will always be more optimal than 'one-size-fits-all' pipelines. Meilisearch solves search-as-you-type better than anyone else, so why compromise? Not to mention that the scalability pattern of BM25 and vector search is entirely different.

This, by the way, is pretty obvious from the fact that you don't publish comparative benchmarks.

There is more relevant benchmark of vector search engines end-to-end, not just algorithms: https://qdrant.tech/benchmarks/

PML is a great collection of implementations, but not the best framework. Also you can use PML with Quaterion: https://github.com/qdrant/quaterion/blob/master/examples/tra...

Starting from 0.5.0 finetuner is no longer an open-source.

> From 0.5.0, Finetuner computing is hosted on Jina Cloud. The last local version is 0.4.1, one can install it via pip or check out git tags/releases here.

But there are some cool ideas implemented there as well, I encourage you to try both!

It is definitely possible to do, if you have a proper training set. You would need to somehow give model a signal, that you are interesting in e.g. arguing style specifically, and not the topic of the text.

The idea is to fine-tune model, which can be used to produce embeddings required for Qdrant. Our design approach is to make things as modular as possible, so you can use framework and engine independently. But we are working on integrating components as well

Check out https://github.com/qdrant/qdrant

I wounder if there is any alternative implementations of ScaNN in languages like Rust.

Is there a reason why this is not accompanied by Hans Zimmer?

I built a very simple tool which generates exercises for any given set of option words.

Source code can be found here https://github.com/generall/Exerciser