Re:Vectors Are the New JSON in PostgreSQL
by williamstein on Monday June 26, 2023 @04:54PM
(#36485708)
I think one of the most important and interesting questions regarding using pgvector is performance, and in particular, how it compares to Qdrant, Weaviate, etc. This post hints at that where it says "I presented a lightning talk called Vectors are the new JSON where I shared use-cases and some upcoming challenges with improving PostgreSQL and pgvector performance for querying vector data. Some problems to tackle (many of which are in progress!) involve adding more parallelism to pgvector, adding support for indexing for vectors with more than 2,000 dimensions, and leverage hardware acceleration where possible to speed up calculations.". But the post doesn't give any numbers. I tried to read the linked lightning talk, but it's on https://www.slideshare.net/, which is a really weird website now -- every few slides it tried to force me to watch a 30 second commercial!?
[ Parent
| on HN ]
I'm a big fan of Postgres, and my app backends are usually very Postgres-heavy. But yeah, I don't get why you'd want to use Postgres for AI inference unless there's some performance reason.
it would be much easier to write query:
select infer_class_from_text_using_ml(some_text) from some_doc
compared to building infra with different services orchestrations.
I'm assuming the cost/effort of getting whatever customized Postgres instance supports efficient ML inference is more than orchestrating something else.
Re:Vectors Are the New JSON in PostgreSQL
by riku_iki on Wednesday June 28, 2023 @12:46AM
(#36502424)
someone has opportunity to prepackage and even sell such product (postgres has BSD license, so you can totally close sources and sell as your own product).
Also, I think there are many cases when smaller models can totally run on CPU.
Re:Vectors Are the New JSON in PostgreSQL
by hot_gril on Wednesday June 28, 2023 @12:42PM
(#36509016)
Right, but they can (and do) also prepackage ML inference services that don't involve Postgres. I think the small model use case would be the biggest reason, same as how I've sometimes used a Postgres table as a cache in a pinch.
Re:Vectors Are the New JSON in PostgreSQL
by riku_iki on Wednesday June 28, 2023 @01:33PM
(#36509760)
> Right, but they can (and do) also prepackage ML inference services that don't involve Postgres.
but then you would need some dataprocessing/warehousing infra integrated to produce dataset for inference and then do something with inference results. Having one db with everything packaged would reduce complexity significantly.
Re:Vectors Are the New JSON in PostgreSQL
by hot_gril on Wednesday June 28, 2023 @02:54PM
(#36510868)
I was just looking at inference, meaning you need some separate infra to do the training either way. If the DB does the training too, then yeah. I think we're pretty far from that, and Postgres by itself might not be the platform for it. There are platforms with SQL-like interfaces that use databases underneath, but there's a lot happening on top.
Re:Vectors Are the New JSON in PostgreSQL
by riku_iki on Wednesday June 28, 2023 @03:39PM
(#36511542)
I think it is all few months work project, given you can run any arbitrary py/c code in extension in postgres, so it is a matter of data wrangling: convert pg table to recordio, run train/inference, convert results back to pg table, wrap logic into pg/py or pg/c functions.