Re:AI chatbots are not a replacement for search engines
by astrange on Monday December 26, 2022 @08:22AM
(#34137411)
> GPT can always generate a bogus answer, because its only notion of whether an answer is good or not is whether your prompt + that answer are statistically similar to text in its training corpus.
No, that is false. “Statistically similar” is doing a ton of work even for the original GPT model - as I said somewhere else, just printing the letter “e” is “statistically similar” to English text, if it can even be grammatically correct you should be surprised.
And it’s certainly not true for the last two generations (InstructGPT and ChatGPT) which weren’t trained with that objective at all but used human raters.
No, that is false. “Statistically similar” is doing a ton of work even for the original GPT model - as I said somewhere else, just printing the letter “e” is “statistically similar” to English text, if it can even be grammatically correct you should be surprised.
And it’s certainly not true for the last two generations (InstructGPT and ChatGPT) which weren’t trained with that objective at all but used human raters.