Those people are wrong; statistics are a kind of reasoning (intuition) and if you try it, it obviously does the other kinds of reasoning too. I have all kinds of answers from it that are logically coherent and can’t possibly have appeared on the internet already.

(Try asking it to write new SCP wiki entries.)

You can see it “does” reasoning if you say “think step by step” after a question. It can attend over its steps then.

What are you basing this assumption on? GPT can always generate a bogus answer, because its only notion of whether an answer is good or not is whether your prompt + that answer are statistically similar to text in its training corpus. The more text from the training corpus it resembles the better GPT will think the answer is. For more obscure topics, this means that the chances of it hallucinating based on similar sounding much more popular topics increases greatly.

This way of working also means that facts that appear more in the training corpus have a higher chance of being part of the model's outputs. For example, if the training corpus was not carefully curated to exclude vaccine conspiracy theories, there would be a good chance that the answer GPT gives to "autism is caused by ___" would be "vaccines", as this would likely be the most common answer in its training corpus. In contrast, Google's algorithms have more sophisticated biasing especially for these kinds of sensitive topics to ensure that more high-confidence sources are likelier to be top results, even if they are a minority of all the pages on the internet on that topic.

No, that is false. “Statistically similar” is doing a ton of work even for the original GPT model - as I said somewhere else, just printing the letter “e” is “statistically similar” to English text, if it can even be grammatically correct you should be surprised.

And it’s certainly not true for the last two generations (InstructGPT and ChatGPT) which weren’t trained with that objective at all but used human raters.