Re:Citation Needed – Wikimedia Foundation's Experimental LLM...
by mattyyeung on Friday May 17, 2024 @07:13AM
(#40388613)
I think you could get a pretty good solution for that using RAG and some tricks with prompt engineering and semantic chunking. With google's very-long-context models (Gemini) you may also have good results simply with some prompt engineering. Preprocessing steps like asking the LLM to summarise themes of each section can be helpful too (in RAG, this info would go in the 'metadata' stored with each chunk, presented to the LLM with each chunk).
A key engineering challenge will be speed ... when you're navigating a document you want a fast response time.
If not, what extra work is needed to bring it to that level?