News for hackers. Stuff that matters.
Re:OpenAI-to-SQLite by Simonw by txtai on Tuesday February 07, 2023 @06:18AM (#34691307)
txtai is an alternative approach to this. It builds a FAISS (also supports HNSW) index alongside a SQLite database. It works with sentence-transformers models. For example, this model https://huggingface.co/sentence-transformers/all-MiniLM-L6-v... is 384 dimensions and works great for semantic search.

https://github.com/neuml/txtai

https://neuml.github.io/txtai/embeddings/query/

Re:OpenAI-to-SQLite by Simonw by foooobaba on Tuesday February 07, 2023 @09:35AM (#34693255)
This looks awesome, thanks for sharing.
Re:OpenAI-to-SQLite by Simonw by txtai on Tuesday February 07, 2023 @09:41AM (#34693356)
Thanks. Just want people to know there are quality FOSS alternatives available for encoding text into embeddings.