News for hackers. Stuff that matters.
Re:Supabase Vector, the Open Source Vector Toolkit for Postgres by kiwicopple on Thursday June 08, 2023 @12:20PM (#36244423)
> It’s important indexes are added after a fair number of vectors have been persisted, which can make some production use cases challenging to implement.

managing indexes are the most challenging part of using pgvector right now. There is a bit of a "goldilocks zone" - you can't create the index with zero data, but if you create it too late then it can take a lot of memory. Andrew (and a few others, like AWS) are working hard on improving this, as well as researching other index types.

Re:Supabase Vector, the Open Source Vector Toolkit for Postgres by deepsquirrelnet on Thursday June 08, 2023 @02:49PM (#36246734)
That’s good to know. I tested it in a docker container and indexed after loading ~1M vectors. Checking EXPLAIN showed it wasn’t using the index at all. I wonder if it’s because it couldn’t load into memory.

Without an index, I found it was too slow for data that size. I didn’t expect issues for data at this “small”, but it’s pretty new to me.