News for hackers. Stuff that matters.
Re:Storing OpenAI embeddings in Postgres with pgvector by simonw on Monday February 06, 2023 @05:51PM (#34685671)
I've been experimenting with something similar to this on top of SQLite.

My experiments so far have involved storing the embeddings as binary strings representing the floating point arrays in a SQLite blob column: openai-to-sqlite is my tool for populating those: https://datasette.io/tools/openai-to-sqlite

I then query them using an in-memory FAISS vector search index using my datasette-faiss plugin: https://datasette.io/plugins/datasette-faiss

Re:Storing OpenAI embeddings in Postgres with pgvector by txtai on Monday February 06, 2023 @06:13PM (#34685922)
You might want to check out txtai (https://github.com/neuml/txtai). It's default configuration is a FAISS index paired with a SQLite database for filtering.

Also worth mentioning that there are plenty of other vector models to try outside of OpenAI. Many open-source and much smaller than 1536 dimensions. Check out the Hugging Face Hub (https://hf.co/models). For example this model (https://huggingface.co/sentence-transformers/all-MiniLM-L6-v...) works great in many cases and is only 384 dimensions. Runs great locally and is FOSS.

Re:Storing OpenAI embeddings in Postgres with pgvector by xrd on Monday February 06, 2023 @07:55PM (#34687004)
Awesome, I came here to ask about SQLite and you already answered it!

I love your writing, Simon. Have you blogged about this yet?

Re:Storing OpenAI embeddings in Postgres with pgvector by simonw on Monday February 06, 2023 @08:04PM (#34687076)
Yeah I wrote these up in https://simonwillison.net/2023/Jan/13/semantic-search-answer... a few weeks ago.