News for hackers. Stuff that matters.
Re:Ask HN: Is RAG the Future of LLMs? by mikecaulley on Sunday April 14, 2024 @06:47PM (#40035188)
This doesn’t consider compute cost; the RAG model is much more efficient compared to infinite context length.
Re:Ask HN: Is RAG the Future of LLMs? by epistasis on Sunday April 14, 2024 @07:00PM (#40035257)
Agreed. I think that RAG implemented via tool-calling, with multiple agents talking to each other, is a much much more likely evolution in the future versus a single unified model.

I could very well be wrong! But we wouldn't want LLMs to be performing lots of arithmetic calculations via exploiting hidden parts of themselves that do linear regression or whatever, far better to just give them the calculator and get results faster and cheaper. Similarly, we can give them a search engine (RAG) and let them figure it out more efficiently.