RAG at scale: Synchronizing and ingesting billions of text embeddings

Re:RAG at scale: Synchronizing and ingesting billions of tex... by losteric on Monday October 09, 2023 @08:27PM (#37827150)

Isn't RAG "just" dynamically injecting relevant text in a prompt? What more would one implement to achieve RAG, beyond using Postgres' built in full text or knn search?

[ Parent | on HN ]

Re:RAG at scale: Synchronizing and ingesting billions of tex... by wanderingmind on Monday October 09, 2023 @08:34PM (#37827225)

what i'm looking for is a neat python library (or equivalent) that integrates end to end say with postgres/pgvector using sqlalchemy, enables parallel processing of large number of documents, create interfaces for embeddings using openai/ollama etc. It looks like FastRAG [0] from intel looks close to what i'm envisioning but it doesnt appear to have integration to postgres ecosystem yet i guess.

[0] https://github.com/IntelLabs/fastRAG

[ Parent | on HN ]

Re:RAG at scale: Synchronizing and ingesting billions of tex... by ddematheu on Monday October 09, 2023 @10:30PM (#37827941)

Through the platform (Neum AI) we support the ability to do this with Postgres, it is just a cloud platform so not a python library.

Curious on what type of customization are you looking to add that you would want something like a library?

[ Parent | on HN ]

Re:RAG at scale: Synchronizing and ingesting billions of tex... by wanderingmind on Tuesday October 10, 2023 @07:31PM (#37839126)

We need something we can orchestrate and control locally and be able make changes if need be. The GUI based interface is good for more mature workflows but our workflows are constantly evolving and requires tweaking that its hard to do with GUI and web interface

[ Parent | on HN ]