RAG is vector search first. It encodes the query, finds nearest vectors in the vector database, retrieves the fragments attached to those vectors, and then sends those vectors to the LLM for it to summarize them.
A general LLM like Gemini or Claude or ChatGPT first produces an answer to a question, based on its training. This doesn't involve searching any external source at that point. Then after that answer is produced, the LLM can try to find sources that match what it has come up with.
Two common alternatives to vector search:
1. Ask the LLM to identify key terms in the user's question and use those terms with a regular full-text search engine
2. If the content you are answering questions about is short enough - an employee handbook for example - just jam the whole thing in the context. Claude 3 supports 200,000 tokens and Gemini Pro 1.5 supports a million so this can actually be pretty effective.
People, we have to stop talking about what we know as though it's all there is. Don't confuse our knowledge for understanding. Understanding only comes from repeadly trying to prove our understandings wrong and learning how things truly are.
It’s bloody useful if you can’t cram your entire proprietary code base into a prompt.
ID: 1
URL: https://example.com/test.html
Text: lksdjflkdsjlksjkl
And then tell it to use the ID to link to the page.
> but that definitely is one particular use of RAG
One use of RAG doesn't imply that all uses of RAG are for grounding LLM hallucinations.
Additionally, I disdain when fallacies come up in conversation when it doesn't appear someone is making an argument in bad faith. We all use logical fallacies by accident from time to time. A good use of calling them out is when they are used in poor taste. I find that more on Reddit than Hacker News.
I have now also engaged in the Tu Quoque fallacy in this reply. See how annoying they are?
> RAG is not used to try to anchor to reality a general LLM by somehow making it come up with sources and links.
but that definitely is one particular use of RAG, i.e. to limit some potential hallucinations by grounding it in data provided in the prompt.