We taught a small LLM to throw away 68% of our RAG context

Re:We taught a small LLM to throw away 68% of our RAG context by chimpanzee2 on Tuesday July 07, 2026 @03:46AM (#48814799)

i wouldn't trust the small llm there.

it will be an intellectual bottleneck when it comes to processing the very information your just arduously fished out of the ocean!!

[ Parent | on HN ]

Re:We taught a small LLM to throw away 68% of our RAG context by kordlessagain on Tuesday July 07, 2026 @08:39AM (#48816886)

I've had good success using a local model for preprocessing (tag extraction) and tuning the search parameters (hybrid search).

[ Parent | on HN ]