Pg_vectorize: The simplest way to do vector search and RAG on Postgres

Re:Pg_vectorize: The simplest way to do vector search and RA... by CuriouslyC on Wednesday March 06, 2024 @12:10PM (#39618137)

It's going to depend on what you're running on, but phi2 is pretty fast so you can reasonably expect to be hitting ~50 tokens a second. Given that, if you are ingesting a 100k token document you can expect it to take 30-40 minutes if done serially, and you can of course spread stuff in parallel.

[ Parent | on HN ]

Re:Pg_vectorize: The simplest way to do vector search and RA... by samier-trellis on Wednesday March 06, 2024 @12:26PM (#39618340)

thanks for the info--good to know we aren't the only ones contending with speed for large documents lol

[ Parent | on HN ]