I'm writing a new vector search SQLite Extension

Re:I'm writing a new vector search SQLite Extension by nnx on Friday May 03, 2024 @02:07AM (#40244625)

Interesting idea.

Do you intend to compress the vector storage in any way and do you intend to implement your own vector search algorithms or reuse some already optimized libraries like usearch?

[ Parent | on HN ]

Re:I'm writing a new vector search SQLite Extension by alexgarcia-xyz on Friday May 03, 2024 @02:43AM (#40244862)

I don't have plans for compressed vector storage. There is support for int8/binary quantization, which can reduce the size of stored vectors drastically, but impacts performance quite a lot. I'd like to support something like product quantization[0] in the future, though!

No plans for using usearch or another vector search library, either. I want to keep dependencies low to make compilingeasy, and I want full control for how vectors are stored on-disk and in-memory.

[0] https://www.pinecone.io/learn/series/faiss/product-quantizat...

[ Parent | on HN ]