News for hackers. Stuff that matters.
Re:I'm writing a new vector search SQLite Extension by alexgarcia-xyz on Friday May 03, 2024 @02:39AM (#40244839)
It's not possible to match pgvector's syntax with SQLite. SQLite doesn't have custom indexes, and since I want to have full control of how vectors are stored on disk, the only real way to do that in SQLite is with virtual tables[0]. And there's a few other smaller details that don't match (postgres operators like <->/<=>, SQLite's limited virtual table constraints, etc.), so matching pgvector wasn't a priority.

I instead tried to match SQLite's FTS5 full text search[1] extension where possible. It solves a similar problem: storing data in its own optimized format with a virtual table, so offering a similar API seemed appropriate.

Plus, there's a few quirks with the pgvector API that I don't quite like (like vector column definitions), so starting from scratch is nice!

[0] https://www.sqlite.org/vtab.html

[1] https://www.sqlite.org/fts5.html

There are no comments on this comment yet.