gravadlax

Hi, I am one of the authors of the post.

We tried using LOPQ, but I don't know the exact parameters used. We probably did not put enough effort into it, though.

Your approach seems interesting. Maybe we will try something similar in the future.

We do have our own custom word (piece) embeddings that we have trained on <good query, bad query>-pairs. There are a few more details about it in https://0x65.dev/blog/2019-12-06/building-a-search-engine-fr....

Just a couple of things:

- We are actually using a weighted average.

- This is just one of the techniques used for recall. The aim at this point is to be super fast and scalable, and not super accurate. Once we reach the precision stage of the ranking (as defined in https://0x65.dev/blog/2019-12-06/building-a-search-engine-fr...), we can afford to do fancier matching.

/ One of the authors

Hi, I am one of the authors of this post.

As you mention, it is a trade-off. In this case we are trying to balance CPU and memory usage. First of all, if you just use the original vectors (or the quantized ones mentioned in the post), it might very well be the case that there is no server where you can even place the index.

So let us consider the example in the post and that we have a server with enough memory. We put the index (1 TB) in RAM and assume that one request takes 1-2 ms. Then a single CPU core can handle 500-1000 requests per second. So now we are using a lot of RAM and very little CPU. Depending on your use case, this can very well be better than what is proposed in the post, but for some it might be preferable to use servers with more balanced resources.

And just to clarify, the memory is shared between CPUs. So you pay the memory price once and you can then scale it to all the CPUs.

Hi, I am one of the authors of this post.

The number 4 billion is made-up. It was chosen in order to illustrate the scale of the problem we are trying to solve. And it also happens to be just below the maximum number possible to store in an unsigned 4 byte integer. This is of course beneficial when trying to build an index with a small footprint. With more queries, we would have to consider using long integers (8 bytes) or some custom type with e.g. 5 bytes.

If you're interested in where the queries come from, you might find these previous posts interesting: https://0x65.dev/blog/2019-12-05/a-new-search-engine.html https://0x65.dev/blog/2019-12-06/building-a-search-engine-fr...