Rag Is a Hack - Hackdot

Re:Rag Is a Hack by bilater on Friday October 06, 2023 @01:16PM (#37793417)

I still don't understand why we have a different term (RAG) for something we've been doing for a while i.e. use a vector db/embeddings coupled with LLM to generate better answers. LLamaIndex (formerly GPTIndex) has been around for a while and the Pinecone boys have been talking about this for months. Is there a breakthrough in terms of the way the pipeline is set up here?

[ Parent | on HN ]

Re:Rag Is a Hack by sprobertson on Friday October 06, 2023 @01:22PM (#37793487)

Because "RAG" is shorter than "use a vector db/embeddings coupled with LLM to generate better answers"

But better answer: "retrieval augmented" has been used in papers since at least 2018 [1], the full "RAG" acronym since at least 2020 [2], and everyone in the GPT3+ hype wave caught up to it once they realized they were doing the same thing.

[1] https://arxiv.org/abs/1808.03430 [2] https://arxiv.org/abs/2005.11401

[ Parent | on HN ]

Re:Rag Is a Hack by jamesbriggs on Saturday October 07, 2023 @04:36AM (#37800065)

Yeah RAG has been around for some time, paper [2] being where I first stumbled onto it — I remember a 2-3 years ago building a RAG pipeline using either the model they trained in paper 2, or that was trained based on the same idea, with Deepset AI's Haystack library. It has been a little surreal to see that terminology become so popular again

[ Parent | on HN ]

Re:Rag Is a Hack by ilaksh on Friday October 06, 2023 @07:10PM (#37797483)

I think the use of this term "RAG" is counterproductive and almost makes me suspicious. Because it is used ambiguously and imprecisely, usually with the main aim being to suggest a level of expertise.

But all it really means is that you put some extra information in your prompt (from somewhere) that didn't come from the user. It's basically the first thing that anyone thinks of once they get the API call working.

People don't use it to only mean vector search, despite often assuming that is what it means.

What I have been leaning towards recently is not using vectors at all, or only using it when I have an extremely close vector match. Instead I give the API one or more functions to look up information that is like an index for documentation or retrieving some info from a database. But it's specific to the application.

I think usually the non-vector stuff is more effective.

[ Parent | on HN ]

Re:Rag Is a Hack by sprobertson on Friday October 06, 2023 @08:54PM (#37798090)

> all it really means is that you put some extra information in your prompt (from somewhere) that didn't come from the user

Yes, and if you are getting (retrieving) some extra information to add to (augment) your prompt for further responses (generation) then it's not very ambiguous.

[ Parent | on HN ]

Re:Rag Is a Hack by ilaksh on Friday October 06, 2023 @10:13PM (#37798448)

Right, the name makes sense. People made it ambiguous because they often believe it means vector search.

I don't think that's correct, but if you look in this discussion, that's the reality of usage. So that makes it ambiguous in practice.

[ Parent | on HN ]

Re:Rag Is a Hack by sprobertson on Saturday October 07, 2023 @01:39PM (#37803760)

I see what you mean - not sure about the usage of ambiguous but some people do have an overly narrow definition in mind

[ Parent | on HN ]