Faster Retrieval for Better, More Efficient RAG

Re:Faster Retrieval for Better, More Efficient RAG by ozkatz on Wednesday November 11, 2026 @02:36PM (#50457489)

all true for a single grep invocation, however a capable LLM can drive grep using similar terms (`grep (red|maroon|scarlet|crimson)`, moving the measure of similarity from embedding/indexing time to inference time. There's benefits to that.

[ Parent | on HN ]

Re:Faster Retrieval for Better, More Efficient RAG by jdthedisciple on Wednesday November 11, 2026 @02:51PM (#50457722)

> red|maroon|scarlet|crimson

still doesn't include typo'ed variants for any of these

[ Parent | on HN ]

Re:Faster Retrieval for Better, More Efficient RAG by ozkatz on Wednesday November 11, 2026 @04:17PM (#50458801)

but a good combination of prompt, harness and model will easily close that gap. a more advanced (but equivalent) form of grep could also offer fuzzy matching/levenshtein distance function to augment that.

[ Parent | on HN ]

Re:Faster Retrieval for Better, More Efficient RAG by c-hendricks on Wednesday November 11, 2026 @04:56PM (#50459239)

This sounds like AOT vs JIT weights.

[ Parent | on HN ]

Re:Faster Retrieval for Better, More Efficient RAG by everforward on Wednesday November 11, 2026 @08:36PM (#50461869)

It will, but it does so by burning tokens. I'm sure you've also seen the output of an overly eager grep by an LLM.

I think we're verging on the point where people will care more about the most token-efficient ways to do a task than the lowest human effort version. Humans are expensive, humans using half as many tokens are cheaper.

[ Parent | on HN ]