And yet, when I google something, am I landing on an experts page, or a seoptimizer who's day job it is to write expert like looking content?

Search is absolutely like asking a random person right now. Whether I get raw hyperlinks back on a page or a chat window, the results are as good as their source data. Garbage in, garbage out.

Google is just as confidently wrong giving me garbage links as chat is as giving me garbage recommendations.

Why are we ok with “AI is equally as bad as the current bad thing?” Even then, it falls flat because the average person doesn’t understand this as well as you do. There will be some severe consequences because of this.

Just because it doesnt solve the garbage in problem, doesnt mean it doesnt solve other problems in the realm of usability.

This is why I am very concerned about people ascribing attributes to LLMs that they simply don't have. The real danger with LLMs, it seems to me, is that people seem to be viewing them as intelligent or as some sort of "truth machines". They are neither. Saying that isn't saying they aren't powerful tools, of course, but a misunderstood tool can be a very dangerous thing.

The difference is that with search you get a whole range of linked resources, and up to now (it may change with AI-generated content going forward) the style of the content usually gives enough signal to make an assessment, as well as comparing content between sites, and you learn which sites to trust.

With ChatGPT, correct and incorrect information is presented identically. You have nothing to go on except falling back to web search to fact-check ChatGPT’s output. This can still be useful as a starting point, but a world with only ChatGPT and no web search would be horrible, would overall be a step back compared to a world with only web search and no ChatGPT.