News for hackers. Stuff that matters.
Re:Powering cost-efficient AI inference at scale with Cloud ... by NBJack on Tuesday November 28, 2023 @09:40PM (#38454788)
LLMs are great at historical data. They however still suck at being kept up to date. I would expect that someone (if not already has) will find a cost efficient way to continuously index new data, but it would be interesting to see how it stacks up with the computing power needed for modern search index pipelines.
Re:Powering cost-efficient AI inference at scale with Cloud ... by deeviant on Wednesday November 29, 2023 @11:09AM (#38461204)
Yeah, when I said "many use cases", that mostly means, "use cases that don't require near-term knowledge." Which are the bulk of my internet searches.

But even know, ChatGPT 4 can use search to get update-to-date info and use it's knowledge store to augment the current information.