I think there is a lot more to it than that. At the root of it is that we are just looking at the product of collective humanity, and not a machine doing any systematic reasoning, like Cyc does.

After all, the bot doesn’t actually come up with original thoughts, but rather it remixes the most thoughtful stuff on the Web. The thing that makes it so powerful is JUST HOW MUCH CONTENT there has been, written by humans, on every subject.

I’ve seen the “mad libs” aspect when I have asked the same prompt multiple times with some variations, as well as examples from sci fi plagiarizing 1950s passages with just a few replacements.

The natural language part fooling most people into thinking they’re talking to someone, I can understand. Because they literally are querying an LLM trained on the Web.

But… the selection of what to train on? Being consistently helpful, and having some personality etc?

I bet it’s not just an LLM. I bet they had a ton of custom training by hundreds of people, and additional algorithms. It’s not just ONE technique producing this. Same with LAMDA etc.

They released GPT-2… does it even come close to following instructions over multiple paragraphs?

>After all, the bot doesn’t actually come up with original thoughts, but rather it remixes the most thoughtful stuff on the Web.

What makes you think a human is any different? I've long believed that humans are not capable of original thought. To truly become creators is beyond us. We just remix previous inputs.

Humans remix things but they also largely use logic - look at what Cyc project does - in order to come up with coherent thoughts and arguments that make sense. They are also open to evaluating what is being told to them, using logic … rather than just continuing with the most “statistically plausible” response, whatever that means.

It is why people who bullshit are considered so low on the intellectual integrity scale. “On Bullshit” is an essay about that.

https://theconversation.com/amp/why-leaders-who-bullshit-are...