News for hackers. Stuff that matters.
Re:AI chatbots are not a replacement for search engines by tsimionescu on Monday December 26, 2022 @04:34AM (#34136127)
> but GPT will never produce a bogus answer when google has the correct one.

What are you basing this assumption on? GPT can always generate a bogus answer, because its only notion of whether an answer is good or not is whether your prompt + that answer are statistically similar to text in its training corpus. The more text from the training corpus it resembles the better GPT will think the answer is. For more obscure topics, this means that the chances of it hallucinating based on similar sounding much more popular topics increases greatly.

This way of working also means that facts that appear more in the training corpus have a higher chance of being part of the model's outputs. For example, if the training corpus was not carefully curated to exclude vaccine conspiracy theories, there would be a good chance that the answer GPT gives to "autism is caused by ___" would be "vaccines", as this would likely be the most common answer in its training corpus. In contrast, Google's algorithms have more sophisticated biasing especially for these kinds of sensitive topics to ensure that more high-confidence sources are likelier to be top results, even if they are a minority of all the pages on the internet on that topic.

Re:AI chatbots are not a replacement for search engines by astrange on Monday December 26, 2022 @08:22AM (#34137411)
> GPT can always generate a bogus answer, because its only notion of whether an answer is good or not is whether your prompt + that answer are statistically similar to text in its training corpus.

No, that is false. “Statistically similar” is doing a ton of work even for the original GPT model - as I said somewhere else, just printing the letter “e” is “statistically similar” to English text, if it can even be grammatically correct you should be surprised.

And it’s certainly not true for the last two generations (InstructGPT and ChatGPT) which weren’t trained with that objective at all but used human raters.