Re:What Is Retrieval-Augmented Generation a.k.a. RAG?
by svcrunch on Friday December 01, 2023 @05:43PM
(#38493576)
I think your comment is accurate, but regarding your last point:
"However, fine-tuning on relevant, high quality, knowledge-rich question/answer pairs seems dominant, when such examples are available or can be generated."
How does one solve the problem of access-controlled data, if not through RAG? Do you imagine a separate version of the LLM for every user, reflecting their unique permissions on the data?
Also, in scenarios where the data is being updated regularly, RAG provides much lower latency to the new information. Deletes also present a challenge for a pure-LLM approach.
Re:What Is Retrieval-Augmented Generation a.k.a. RAG?
by willy_k on Friday December 01, 2023 @08:19PM
(#38494831)
I don’t know much about the space, but couldn’t LoRA be used to (moderately) efficiently use a relevant and individual fine-tuned model alongside a general LLM?
"However, fine-tuning on relevant, high quality, knowledge-rich question/answer pairs seems dominant, when such examples are available or can be generated."
How does one solve the problem of access-controlled data, if not through RAG? Do you imagine a separate version of the LLM for every user, reflecting their unique permissions on the data?
Also, in scenarios where the data is being updated regularly, RAG provides much lower latency to the new information. Deletes also present a challenge for a pure-LLM approach.