News for hackers. Stuff that matters.
Re:Embeddings: What they are and why they matter by freediver on Wednesday October 25, 2023 @02:07AM (#38009662)
If you are looking for lightweight, low- latency, fully local, end-to-end solution (chunking, embedding, storage and vector search), try vectordb [1]

Just spent a day updating it with latest benchmarks for text embedding models.

[1] https://github.com/kagisearch/vectordb

Re:Embeddings: What they are and why they matter by tayo42 on Wednesday October 25, 2023 @10:55AM (#38013483)
It looks like it does sliding window on tokens? That works well, you don't slice it in the middle of sentences?