Re:Why HNSW is not the answer and disk-based alternatives mi...
by generall on Tuesday December 24, 2024 @03:22AM
(#42500499)
In IVF you can start checking conditions only in the final bucket. There are no guarantees if the bucket has any acceptable value, and there are no procedures to find the bucket which has acceptable values before scanning it.
Re:Why HNSW is not the answer and disk-based alternatives mi...
by VoVAllen on Tuesday December 24, 2024 @04:45AM
(#42500817)
I'm not sure I fully understand your point. This can be implemented quite easily by organizing each posting list with specific attributes to locate the values efficiently. For example, you could build each posting list into a B-tree, or similar to a column-oriented format that stores attribute statistics to enable skip scanning. Could you elaborate more?
[1] https://arxiv.org/abs/2403.04871