I suspect this is due to a fundamental misunderstanding of what tools like ChatGPT actually are, what their capabilities are, etc. I think there is a large population of people who think they're simply more sophisticated versions of Siri. In fact, I'd go so far as to say that the vast majority of people don't even realize they're powered by different things, but instead just see the "marketing term currently known as AI" as a singular monolithic entity, and all related technology just gets lumped into the same category. That's miles away from understanding how "use a language model to interpret what I'm asking you to search for, and then plug that into a normal search engine, and read me back the most relevant result", is fundamentally different from "use a language model to predict the stastically most likely next sequence of tokens based on the ones you gave me."

I don't mean this as a way of thumbing my nose at people who don't know this stuff, but rather, to point out what a massive failure the completely nonexistent consumer education surrounding ML products has been.

> I genuinely don't understand the appeal of AI for search

If you're good at googling the flow is: Ask the question > Clock which result isnt spam and click it > Figure out how to dismiss the cookies gate without accepting the cookies > Dismiss the google login box > Dismiss the popover pushing you to install an app > Scroll the page or ctrl+F to find the answer

With ChatGPT it's just type your question and your answer is appearing right away.

Of course I understand ChatGPT shouldn't be used for this because it will lie to you and make things up. But I am saying that's why you'll see people who don't know that preferring it over Google.

painful how accurate that is

Aaand this is basically the story of silicon valley over and over again.

Build a product that's more convenient, and people will use it. Google is so full of shit now, and even answer boxes are below 4 ads, it's just way more comfortable and efficient to ask chatgpt.

This morning I wanted to know how many calories there are in a breaded chicken breast. Chatgpt told me in 3 seconds after asking. Google would have been way more time. (sidenote: I also hate google, so I'd much rather use chatgpt anyway)

Also Google (and distressingly, DDG as of late) sanitizes the hell out of the SERPs to only present you with “approved viewpoints”.

And I’m not talking about fringe Q Anon type stuff: the other day I was looking up the specifics of China’s “Blue Sky Initiative” climate change policies and the only thing Goog/DDG would show me (despite several attempts at rephrasing the query) was Western industry think tanks bellyaching about how the policies effect profits. It took me a good ten minutes of refining my search before I got an English translation of the actual policy bullet points.

I can’t imagine this being such a PITA on 2013 era Google.

out of curiosity, how did you verify the information?

I didn’t, but it sounded right. I know it might be incorrect.

However, I’ve seen plenty of bad information in google answer boxes too. And finding it in actual search results is going to be way more time. It’s not a life and death question.

My googling flow: Ask the question > Clock which result isnt spam and click it > uBLock Origin blocks everything > ctrl+F to find the answer

It lies like crazy on surprising things. Database parameters for an enterprise provider, for example, I've seen hallucination in 5% of cases. That's _bad_ when its taken as authoritative.

I like that you can get results quickly and it solves the "I don't know what I'm looking for" problem.

I'm learning typescript and ran into a weird typescript construct the other day, I threw it into chatgpt and asked "what is this" and it explained it to me. I'm not entirely sure if pasting in a bunch of braces and parentheses into google would find me the same result.

Don't get me wrong, thats a great use case!

None of the results are surprising if you simply read what OpenAI writes about their own machine. It's an autocomplete engine.

An autocomplete engine with "sparks of AGI", because gotta get that sweet sweet VC money and market it as the Next Big Thing.

Well, if you make up the answer from nowhere yourself, you save the work of creating a question and typing it on the ChatGPT prompt.

I mean the internet lies and makes stuff up too, being first on Google results has nothing to do with truthfulness.

That's the glory of a search engine. You have multiple sources available to review, compare and - with enough experience - learn which sources you can more readily trust than others.

AI offers none of the above. Ask the same question twice in a row, maybe you'll get a different answer, maybe you won't. You won't know which of the two answers are hallucinations, which are true factoids it trained on, and which are bogus things it had been trained on. There's literally no providence for the results- no trail, no references, nothing, because it's really a sophisticated game of "Whose Line" where everything's made up and nothing matters.

It's funny that you mention recipes at the end because that's exactly what I like to use ChatGPT for. Every single recipe site has been so corrupted by SEO that you need to scroll past 12 paragraphs of nonsense to get to the actual recipe, and more often than not once you actually get to the recipe part you'll be bombarded with popups about newsletters and cookies or some late-loading ad will cause the view to shift past the step you're trying to look at.

ChatGPT is great for distilling that experience down to just a simple recipe for whatever it is I'm looking for.

Tangentially: props to AnyList for their amazing plugin that scrapes recipes off of sites like this and stores them in an easy-to-use format.

I think it's pretty safe to say if any of these AI-based models gain dominance, they will eventually add junk to their output if that junk makes money.

My prediction: By 2025, you'll be able to purchase sponsored sentences and sponsored paragraphs in ChatGPT's (or whoever's on top) output.

I just used ChatGPT4 this past weekend to come up with smoothie recipes and then a shopping list to take to Whole Foods. Can also give it ingredients you already have and ask for recipes. It’s a killer application for me ha

ChatGPT like AI is the natural next step in the arms race of SEO vs Search. Right now is a mechanism to fight SEO. It will continue to work for a few years.

When you're not sure what to Google (such as when working in an unfamiliar domain), asking ChatGPT for help and then researching it's answer with normal searches is very effective.

This is the logical endgame for the search engine form factor. Remember watching mom ask google questions using complete sentences and punctuation with a “please and thank you”?

The appeal of a conversational response was too great to wait for a solution to the problem of training data quality.

The appeal is that now regular search engines are so bad at giving you useful content that using a LLM is now the equivalent of "google-dorking" to find relevant information.

Only because the LLM didn't have any AI generated blogspam to get trained on. That's going to change very quickly.

Any non-trival LLM that works by scraping the internet is already sufficiently advanced to be able to classify blogspam.

The issue Google has with blogspam is they don't want to filter it because it inevitably uses Google's advertising. So Google gets to self-deal traffic to blogspam which makes them money. They're not incentivized to actually eliminate blogspam.

LLMs aren't automatically disincentivized from training on blogspam so they're not going to avoid it either.

The appeal is that AI answers your question. Maybe a wrong answer. Maybe from questionable sources.

Search, on the other hand, doesn't answer your question.

Simple: get the search results, have LLM read and evaluate them.

It's devilishly difficult to get citations in there, you're not going to do it with langchain, but its possible (c.f. Bing).

Python x langchain x LLMs makes it very easy to create demos so there's been an initial influx of meh stuff, I'm very excited for 6 months from now.

A niche provider that doesn't do generative AI very well but does "AI powered search with citations" is perplexity.ai

I've used for rather obscure queries and liked the summary the AI wrote as well as the links to dive deeper. I imagine that's what AI search will look like across all providers before long.

That site looks incredibly good.

I threw in a question Wikipedia does kinda answer but leaves a lot of details off, and it enumerated a set of possible answers, with my favored option linking into a forum where it looks like less than a dozen people post, but with people that tried all variations of it and know all of the details.

DDG and Google would never show me that site (yeah, I've tried).

Just to add, that forum has better answers than the ones GPT created directly on the search engine. If it was a "consult the Oracle" interface instead of a search engine, it wouldn't be useful at all.

> I genuinely don't understand the appeal of AI for search. The provenance of information is just as important as the resulting information for pretty much anything I search for.

Search requires work on the part of the user to distinguish between good links and bad. AI is an oracle just tells you what you're looking for.

Now you and I might think this is a terrible way to evaluate the veracity of information. But think about the new generation of mobile-native users who were raised on simplistic discourse in tweet-length messages, and would rather watch a 1-min video on a topic than scan search results for 30 seconds.

For this group, searching for information where more than 2 clicks is required is going to be a "too complicated", anda "bad user experience".

> AI is an oracle just tells you what you're looking for.

LLMs are oracles that arrange words in a probabilistic order that are grammatically correct and may be factually correct. Unfortunately there's no way to evaluate the probability of confabulation with any of the LLM chat bots. The distribution of occurrences confabulation is also not regular or predictable nor is it fixed. So you can't ever say "ChatGPT is bad at X" because it can be bad today, good tomorrow, then bad the next day.

It sounds like we are dooming an entire generation by catering to a maladaption of technology-addiction induced ADD.

Idiocracy and The Machine Stops ... All At Once.

LLMs are the worst method for information retrial, except for every other method we tried.