But I do agree that adding another level of fake news generation is a solution in desperate need of a problem.

Getting a completely untrustworthy, unsourced response seems worse than useless. Google has been going this way for a while, with its instant answers or whatever, but at least those try to cite a search result and you can read the surrounding context which Google got the result from.

A few sources will control the information we get in a much more direct and extreme way than now, that conscious skepticism will no longer be able to defend. Whatever handwaved promises we get now will be gone ten years from now.

If there wasn't such a gee-whiz coolness factor about conversational search results distracting us, we'd never tolerate that in principle.

There's nothing fundamentally new about this. The average person is blissfully unaware of the conversations being had between powerful individuals, PR experts, producers, and so on about how said person should be manipulated.

Misinformation and manufactured narratives are omnipresent and all we can do is consume as diverse a media diet as possible and develop a good nose for bullsh*t.

I know there are expert vetted information sources, but you generally have to pay for the quality and they do not get linked to from Bing and Google.

Expert vetting doesn't even touch the underlying problem, because the pursuit is not expertise, in and of itself, but objectivity. And that's something far scarcer than expertise, and increasingly fleeting in today's world. A Chinese expert is probably going to have a different perspective on e.g. the Uyghurs than an America expert on such, even if both are in no way trying to mislead but giving their most sincere analysis of the situation.

Even on topics that are not conventionally controversial, you'll find a similar issue. Ask two astrophysicists of different worldviews on dark matter, and you are going to get two very different answers that, in many ways, will be incompatible. Simply "believing" one over the other doesn't really make any sense, nor does randomly polling astrophysicists and taking that as the definitive truth.

My point was even a step before this, that even getting the consensus facts correct is a major challenge when the internet is written by children, bored volunteers, mechanical Turk contributors from across the world, adversarial actors, and content producers churning clickbait. Having a professional journalist investigate a topic, then have a separate professional fact-check, and yet another professional edit all for a publication that is trying hard to maintain a reputation and will publish retractions if necessary is all miles better as a starting point for determining truth, but sadly that cultural activity is nearly dead.

So why do companies like Microsoft and Google push in this direction? Why are they making the results more and more opaque? You'd hope that they would care enough to be good stewards of the power granted to them through their information monopoly, but barring that, you'd hope that they'd recognize that people want results they can verify, not just random answers.

Or maybe they're hoping that people don't care about verifying results, hoping that people just want an answer that's not necessarily the right answer? It seems like a dangerous gamble.

In this case, Bing AI will operate very differently from ChatGPT.

"(Fact that isn't true)[Source that does not make that claim but comes close enough that you wouldn't notice it just by skimming]"

leading to even more people thinking its true than otherwise.

Yet the media tends to miss the internationally understated nuance and run with the claims at face value, which an LLM will then pick up on and state, "Yes scientists have proven that [x] does cause [y]. [1][2][3][4][5]" That claim then gets repeated elsewhere, and eventually that "elsewhere" goes on to become part of the LLM's new training material where it's basically training off its own output.

It'd be ironic if something that's ideally designed to make the breadth of human knowledgeable more readily available and accessible than ever before, ends up just making society vastly more misinformed than ever before.

Existing Google searches are polluted with false information, and Google’s has been losing that battle. It’s probably not even possible to win.

So rather than saying search engines should always be perfectly accurate and errors are catastrophic, we should accept that search engines are, and have always been imperfect, and need to give us enough info to validate facts for queries important enough to merit it.

When I use Google for research, I get articles written for SEO to push products and often have to refine and refine and refine to get something useful, which I then can follow up by googling to learn more. With difficulty.

Honestly I don't know how much I'd use ChatGPT if I had the internet of 2016 and Google.

For example, until a few months the results for "pork cooked temperature" and "chicken cooked temperature" were returning incorrect values, boldly declaring too low of a temperature right at the top of the page (I know these numbers can vary based on how long the meat is at a certain temperature, but I verified Google was parsing the info incorrectly from the page it was referencing, pulling the temperature for the wrong kinds of meat). This was potentially dangerous incorrect info IMO

Yes, so would I. And I also double check things like Google Maps -- a tool I find very helpful but don't trust blindly. But... do most people think to take a close look at Google Maps to make sure it makes sense, and trust their own judgement if they disagree with the map? Will most people fact check confident LLM outputs?

What is ridiculous is, when, say, Stack Overflow has a good answer, it is a few lines down or on the next page in the search results, but some page-mill SEO site is in snippets up top with a completely wrong or naively pathetic partially correct answer. It is so annoying it has lowered my opinion on Google a lot in recent times.

Ad content invariably gets vetted by humans. The fact that it shows up in the ad demonstrates human failures more than failures of LLMs.

This is scary to read. You always need to fact-check the results, whether they come from a search engine, an AI, or a primary source!

Fine. We need another good winter or ten before we decide we want to commit societal suicide via deepfake tsunami.

Genuine, honest question: How did you come to the belief that search engines are reliable sources of truth?

I completely agree that search engines provide a valuable service. But in my own work, I find them to very often point to inaccurate information, sometimes greatly so. I don't think this is terribly surprising, given Sturgeon's law, but still.

Google's branding frames itself as the expert in the novice-expert problem. The vast number of users implicitly take on the role of the novice by virtue of using the product. They've already self-identified as a novice which makes both parties complicit in the arrangement.

And this stance seriously hasn't bitten you in your life or career to date?