> But the distinction is it's less rude, and more knowledgeable.

And that regular people assume it basically is an oracle, which doesn't happen to many people online

The internet is full of people that deliver answers confidently and eloquently enough to be widely believed, especially if they have been right on other topics. They even have similar feedback loops to GPT in learning what sort of answers impress other forum users.

I'm not saying the people that think ChatGPT is an oracle don't exist, but I think it probably has more people in the surprised it works at all camp, and certainly more people inclined to totally disbelieve it than a random off Quora or Reddit...

> The internet is full of people that deliver answers confidently and eloquently enough to be widely believed

Why should that be the baseline? After all you usually have hundreds or thousands of other people who have made their opinions on the subject publicly accessible, which more or less solves this problem most of the time (and I’m not talking about random people on Quora and their bizarrely absurd answers..)

When someone is wrong on the internet, nailing both the tone and vocabulary of the type of expert said wrong someone is purporting to be is rare and impressive. But ChatGPT nails both an overwhelming amount of the time, IME, and in that way it is entirely unlike a person on the internet.

Exactly. Most wrong-on-the-internet people have tells. And most of the rest have history. We all use this all the time on HN. But with LLMs the former gets engineered away and the latter is nearly useless. Except of course to the extent that we say,"Wow, ChatGPT is absolutely untrustworthy, so we should never listen to it." But given how many people are excited to use ChatGPT as a ghostwriter, even that is limited.

You can also ask a person on the internet to source their information and/or provide context.

> It's like a person on the internet -- in that it's wrong 20% of the time, often confidently so. But the distinction is it's less rude, and more knowledgeable.

Do you think if it is trained on only factual content, it will only say factual things? How does that even really work? Is there research on this? How does it then work for claims that are not factual, like prescriptive statements? And what about fiction? Will it stop being able to write prose? What if I create new facts?