This is akin to a HN comment asking someone to search the Internet for something on their behalf, while discussing search engine algorithms!
RAGAS: RAG Assessment [2]
RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrieval [3]
Self-RAG: Self-Reflective Retrieval-Augmented Generation [4]
Agentic RAG: Agentic Retrieval-Augmented Generation [5]
GraphRAG: Graph Retrieval-Augmented Generation [6]
[1] https://docs.haystack.deepset.ai/docs/hypothetical-document-...
[2] https://docs.ragas.io/en/stable/
[3] https://arxiv.org/html/2401.18059v1
[5] https://langchain-ai.github.io/langgraph/tutorials/rag/langg...
[6] https://www.microsoft.com/en-us/research/blog/graphrag-unloc...
This is the new “can you Google that for me?”
RAPTOR for example is a technique that groups and clusters documents together, summarizes them, and creates embeddings defining a sort of a Tree. Paper: https://arxiv.org/html/2401.18059v1
Agentic RAG is creating an agent that can decide to augment "conversations" (or other LLM tools) with RAG searches and analyze its relevance. Pretty useful, but hard to implement right.
You can google the others, they're all more or less these "techniques" to improve an old-fashioned RAG search.
- Hybrid Retrieval (semantic + vector) and then LLM based Reranking made no significant change using synthetic eva-questions
- HyDE decreased answer quality and retrieval quality severly when measured with RAGAS using synthetic eval-questions
(we still have to do a RAGAS eval using expert and real user questions)
So yes, hybrid retrieval is always good - that's no news to anyone building production ready or enterprise RAG solutions. But one method doesn't always win. We found semantic search of Azure AI Search being sufficient as a second method, next to vector similarity. Others might find BM25 great, or a fine tuned query post processing SLM. Depends on the use case. Test, test, test.
Next things we're going to try:
- RAPTOR
- SelfRAG
- Agentic RAG
- Query Refinement (expansion and sub-queries)
- GraphRAG
Learning so far:
- Always use a baseline and an experiment to try to refute your null hypothesis using measures like RAGAS or others.
- Use three types of evaluation questions/answers: 1. Expert written q&a, 2. Real user questions (from logs), 3. Synthetic q&a generated from your source documents