News for hackers. Stuff that matters.
Re:Supabase Vector, the Open Source Vector Toolkit for Postgres by jonplackett on Thursday June 08, 2023 @10:26AM (#36242398)
Anyone have any experience or knowledge of this VS pinecone?
Re:Supabase Vector, the Open Source Vector Toolkit for Postgres by AdamProut on Thursday June 08, 2023 @10:37AM (#36242588)
This may depend on if your use case is only vector search in isolation (ANN lookups). In this scenario PG Vector is potentially not the best option (https://ann-benchmarks.com/)

That said, using PG Vector (or using other SQL databases with vector search support.. many have this capability) will let you do both ANN and your usual SQL filtering and joining (or full text, or.. etc) to produce more hand tuned results to a query. This is something the specialized vector databases don't have much support for.

Re:Supabase Vector, the Open Source Vector Toolkit for Postgres by mritchie712 on Thursday June 08, 2023 @10:33AM (#36242507)
we're[0] using pgvector via supabase in prod and we've had 0 issues. We store things like the natural language explanation of what a SQL statement does and use pgvector to look up the SQL based on a users new natural language question.

I think the argument against pgvector would be "it doesn't scale", but we're using it against hundreds of thousands of rows of data and the performance is solid (< 100ms). Maybe we'll run into issues with tens of millions of rows, but we'll worry about that when it actually becomes a problem.

0 - https://www.definite.app/

Re:Supabase Vector, the Open Source Vector Toolkit for Postgres by aligajani on Thursday June 08, 2023 @10:36AM (#36242568)
How do you prevent SQL injections or bad queries?
Re:Supabase Vector, the Open Source Vector Toolkit for Postgres by mritchie712 on Thursday June 08, 2023 @11:03AM (#36243032)
we only execute SELECT statements and the user we connect with has readonly access.

On "bad queries", we rely heavily on the users past SQL history to understand correct JOIN's and how they normally define metrics / use the database in general.

Re:Supabase Vector, the Open Source Vector Toolkit for Postgres by roseway4 on Thursday June 08, 2023 @10:56AM (#36242919)
We embed Postgres and pgvector in Zep[0], a memory store for LLM apps. It’s rock solid, even with very wide vectors, and with the right indexes ANN is performant up to tens of millions of rows. Haven’t tested beyond this, though.

It’s super useful for hybrid search: vector similarity search over unstructured data joined with filtering by structured data.

One caveat: It’s important indexes are added after a fair number of vectors have been persisted, which can make some production use cases challenging to implement.

[0] https://github.com/getzep/zep

Re:Supabase Vector, the Open Source Vector Toolkit for Postgres by brigadier132 on Thursday June 08, 2023 @11:36AM (#36243667)
When you say you can do vss along with filtering. Does the filtering happen before or after the vss?
Re:Supabase Vector, the Open Source Vector Toolkit for Postgres by roseway4 on Thursday June 08, 2023 @12:18PM (#36244395)
Logically at the same time in the sql query. This may not be how the query is executed by the database, though, and is up to the optimizer.
Re:Supabase Vector, the Open Source Vector Toolkit for Postgres by kiwicopple on Thursday June 08, 2023 @12:20PM (#36244423)
> It’s important indexes are added after a fair number of vectors have been persisted, which can make some production use cases challenging to implement.

managing indexes are the most challenging part of using pgvector right now. There is a bit of a "goldilocks zone" - you can't create the index with zero data, but if you create it too late then it can take a lot of memory. Andrew (and a few others, like AWS) are working hard on improving this, as well as researching other index types.

Re:Supabase Vector, the Open Source Vector Toolkit for Postgres by deepsquirrelnet on Thursday June 08, 2023 @02:49PM (#36246734)
That’s good to know. I tested it in a docker container and indexed after loading ~1M vectors. Checking EXPLAIN showed it wasn’t using the index at all. I wonder if it’s because it couldn’t load into memory.

Without an index, I found it was too slow for data that size. I didn’t expect issues for data at this “small”, but it’s pretty new to me.

Re:Supabase Vector, the Open Source Vector Toolkit for Postgres by sodiumSalt on Thursday June 08, 2023 @02:21PM (#36246331)
Zep seems really cool! I think hybrid search is a great paradigm and believe it will continue to grow. How do you guys plan on dealing with data deletion requests and other GDPR rules around storage?
Re:Supabase Vector, the Open Source Vector Toolkit for Postgres by roseway4 on Thursday June 08, 2023 @02:41PM (#36246600)
* * *
Re:Supabase Vector, the Open Source Vector Toolkit for Postgres by kiwicopple on Thursday June 08, 2023 @12:15PM (#36244341)
for those who jump straight to the comments (like me), this is a python for managing unstructured vectors/embeddings in Postgres using pgvector. It works with any Postgres/pgvector database

The core of the API is this:

    import vecs

    DB_CONNECTION = "postgresql://<user>:<password>@<host>:<port>/<db_name>"

    # create vector store client
    vx = vecs.create_client(DB_CONNECTION)

    # create a collection of vectors with 3 dimensions
    docs = vx.create_collection(name="docs", dimension=3)
    
    # Add embeddings
    docs.upsert(vectors=[("vec0", [0.1, 0.2, 0.3], {"year": 1973})])

    # Query embeddings
    docs.query(query_vector=[0.10,0.21,0.29], limit=1)
This stores all the data in a new `vecs` schema in your database. It took me a while to grasp the nature of structured (eg, manage tables/data with migrations) with unstructured/nosql, but the use cases for vectors are very often geared towards data scientists/engineers. There is some more info about the interop between these approaches here: https://supabase.com/docs/guides/ai/structured-unstructured