Same can be said about the results that pop up on your favorite search engine or asking other people questions.

If anything advances in AI & search tech will do a better job at providing citations that agree & disagree with the results given. But this can be a turtles all the way down problem.

I believe it’s turtles all the way down.

Maybe I’m just from a different time but I can recall hearing and respecting the advice, “don’t believe everything you read on the internet”.

So, given a web page (found via search, example 1 in your comment), I remember that advice and take any suggestions with a grain of salt.

And if I’m using a social platform (I’ll reference reddit since it has historically been useful [in my experience] for interacting with other humans), typically / hopefully the useful information provided by the individual will preface their suggestions with IANAL (I’m not a lawyer) or ‘in my experience’.

In both cases, the suggestion to the query is prefaced with “this might not be right, but…”.

With my limited understanding of “AI” options that are available (to date), hinting at the fact that the information provided might not be accurate is non-existent. The entire perceived value in these systems is that the information is accurate and reliable. But that isn’t 100% the case.

So, hopefully, people will continue to take any advice with a grain of salt but it’s hard to imagine they will when the entire concept behind the “AI” idea is that the information provided is accurate.

I’ll conclude with this statement, which might negate any agreement with my statement thus far and show my disdain, but I’m not a fan of this ignorant embrace of “AI”. IMHO, it (“AI”) is embracing the idea that there are simple solutions to complex problems. I don’t think, 99% of the time, there are simple solutions to complex problems and pretending like there are is only going to create more complex problems.

Maybe I’m just being ignorant and very negative about the advances in tech but I can’t help but think folks are setting things up for failure and more confidently incorrect information.

No it won't and random search popup results are already a massive societal problem (and they're not even used like people are attempting to use AI - to make decisions over other peoples lives in insurance, banking, law enforcement and other areas where abuse is common when unchecked).

There’s a real difference in scale and perceived authority: false search results already cause problems but many people have also been learning not to blindly trust the first hit and to check things like the site hosting it.

That’s not perfect but I think it’s a lot better than building things into Word will be. There’s almost no chance that people won’t trust suggestions there more than random web searches and the quality of the writing will make people more inclined to think it’s authoritative.

Consider what happened earlier this year when professor Tyler Cowen wrote an entire blog post on a fake citation. He certainly knows better but it’s so convenient to use the LLM emission rather than do more research…

https://www.thenation.com/article/culture/internet-archive-p...

Low quality blogs etc stand out as low quality, LLMs can eloquently state truths with convincing sounding nonsense sprinkled through out. It's a different problem and many people already take low quality propaganda at face value.

I think this is a failure in how we fine-tuned and evaluated them in RLHF.

"In theory, the human labeler can include all the context they know with each prompt to teach the model to use only the existing knowledge. However, this is impossible in practice." [1] Therefore causing and forcing some connections that are not all there for the LLM. Extrapolate that across various subjects and types of queries and there you go.

1:https://huyenchip.com/2023/05/02/rlhf.html