A Comprehensive Guide for Building Rag-Based LLM Applications

Re:A Comprehensive Guide for Building Rag-Based LLM Applicat... by chandureddyvari on Thursday September 14, 2023 @07:24AM (#37507487)

Yeah what you mentioned might be true. Currently our understanding on how LLMs really work behind the screens is limited. For example, there was a recent research[1] where LLM's accuracy is better if the context is added at the beginning when compared to the end of the prompt. So it's mostly by trial & error to figure out what works out best for you. You can use FAISS or similar to have the embeddings in-memory instead of a full fledged vector DB. But pg vector is convenient plugin if you already have postgres instance running

[1]- https://towardsdatascience.com/in-context-learning-approache...

[ Parent | on HN ]

Re:A Comprehensive Guide for Building Rag-Based LLM Applicat... by halflings on Thursday September 14, 2023 @02:36PM (#37512916)

What I mentioned doesn't depend on how LLMs work, the end result is the same (retrieving useful inputs to pass to your LLM). Just meant that a lot of people can just do this in-memory or in ad-hoc ways if they're not too latency constrained.

[ Parent | on HN ]

Re:A Comprehensive Guide for Building Rag-Based LLM Applicat... by bayesian_limit on Tuesday September 19, 2023 @12:01PM (#37571917)

Zilliz just published an article comparing QPS (queries per second) with pg vector vs. Milvus. The results are clear - Milvus, a database designed ground-up for handling vector indexes, outperformed in terms of speed and latency. Dive into the details here. https://zilliz.com/blog/getting-started-pgvector-guide-devel...

Full disclosure, I just joined Zilliz this week as a Dev Advocate.

[ Parent | on HN ]