Korvus: Single-Query RAG with Postgres

Re:Korvus: Single-Query RAG with Postgres by simonw on Thursday July 11, 2024 @04:27PM (#40940333)

Does this work my running LLM such as Llama directly on the database server? If so, does that mean that your database and the LLM are competing for the same CPU and memory resources?

Can it run the LLM on a GPU?

[ Parent | on HN ]

Re:Korvus: Single-Query RAG with Postgres by smarvin2 on Thursday July 11, 2024 @05:49PM (#40940906)

It does work by running the LLM on the database server but you can configure the LLM to run on the GPU

[ Parent | on HN ]

Re:Korvus: Single-Query RAG with Postgres by ekianjo on Friday July 12, 2024 @06:44AM (#40944343)

if you are using your database extensively how do you scale up your GPU resources for korvus?

[ Parent | on HN ]