News for hackers. Stuff that matters.

A comment on The LLM RAG Obituary: Killed by Agents, Buried by Context Windows. Parent comment

Re:The LLM RAG Obituary: Killed by Agents, Buried by Context... by jgalt212 on Wednesday October 01, 2025 @09:00PM (#45445374)
> Grep works great when you have thousands of files on a local filesystem that you can scan in milliseconds. But most enterprise RAG use cases involve millions of documents across distributed systems

Great point, but this grep in a loop probably falls apart (i.e. becomes non-performant) at 1000s of docs, not millions and 10s of simultaneous users

[ Parent | on HN ]
Re:The LLM RAG Obituary: Killed by Agents, Buried by Context... by nbstme on Wednesday October 01, 2025 @09:03PM (#45445389)
Why does grep in a loop fall apart? It’s expensive, sure, but LLM costs are trending toward zero. With Sonnet 4.5, we’ve seen models get better at parallelization and memory management (compacting conversations and highlighting findings).
[ Parent | on HN ]
Re:The LLM RAG Obituary: Killed by Agents, Buried by Context... by adrianbooth17 on Thursday October 02, 2025 @03:30AM (#45447140)
"LLM costs are trending toward zero". They will never be zero for the cutting edge. One could argue that costs are zero now via local models but enterprises will always want the cutting edge which is likely to come with a cost
[ Parent | on HN ]
Re:The LLM RAG Obituary: Killed by Agents, Buried by Context... by jgalt212 on Thursday October 02, 2025 @08:27AM (#45448759)
If LLM costs are trending towards zero, please explain the $600B openai when Oracle and the $100B deal with Nvidia.

And if you think those deals are bogus, like I do, you still need to explain surging electricity prices.

[ Parent | on HN ]
Re:The LLM RAG Obituary: Killed by Agents, Buried by Context... by flyinglizard on Thursday October 02, 2025 @11:54AM (#45451368)
They're not trending toward zero; they're just aggressively subsidized with oil money.
[ Parent | on HN ]