Every story and comment from the Hacker News API, in a layout borrowed from 2001.
A comment on Ask HN: How do I train a custom LLM/ChatGPT on my own documents in Dec 2023?. Parent comment
Things get messy when the number and type of documents increase. Below are the reasons why you may need advanced RAG.
1. Intelligent Data Parser 2. Chunking efficiently 3. Choice of embedding models 4. Query transformation 5. RAG technique 6. Prompt design 7. Feedback loop
Check out my blog on the 27 parameters, considerations and techniques one could follow to build a State-of-the-Art Chatbot.
https://www.lyzr.ai/27-parameters-techniques-considerations-...
So here is the quick guide.
For simpler usecases - inbuilt vector database RAG is sufficient For more complex ones - LlamaIndex or Langchain options are suitable For enterprise grade production use cases - Lyzr's SOTA RAG architecture comes in handy
There are no comments on this comment yet.
Things get messy when the number and type of documents increase. Below are the reasons why you may need advanced RAG.
1. Intelligent Data Parser 2. Chunking efficiently 3. Choice of embedding models 4. Query transformation 5. RAG technique 6. Prompt design 7. Feedback loop
Check out my blog on the 27 parameters, considerations and techniques one could follow to build a State-of-the-Art Chatbot.
https://www.lyzr.ai/27-parameters-techniques-considerations-...
So here is the quick guide.
For simpler usecases - inbuilt vector database RAG is sufficient For more complex ones - LlamaIndex or Langchain options are suitable For enterprise grade production use cases - Lyzr's SOTA RAG architecture comes in handy