We've been able to produce that kind of speech before though. I wrote programs using Markov Chains 10 years ago which could produce almost real sounding speech but I didn't think it was intelligent.

The GPT models are essentially massive databases along with the use of probability models for how words are connected. Do you think that means intelligence? Can it generate new knowledge of things outside its database? Can it generate novel theories for things outside its understanding?

The GPT models often generate streams of text which can found word for word in its database, i.e. it's not really generating anything new as just mimicking. When a parrot repeats a human, it doesn't understand what the words mean, it's just making sounds.

But at some level our own brains are just very large databases and very good prediction engines, at least when running on auto-pilot and lacking any deep understanding. Hence the phenomenon of BS artists. Sad to say but I've heard speeches that were less coherent and interesting than what GPT-3 produces.

Where in the GPT models is the conscious (or even subconscious) thinking, comprehension, understanding, relating to experience, imagination etc, etc, you know: actual intelligence as a human would feel and use it, general or otherwise?

Tic Tac Toe, Chess, Image Classifying, Translation, Go, Sentence Construction, etc.; We are creating phenomenally impressive calculating worm-equivalents, but nothing human-level, or even remotely so.

Where in the human mind are these things? We judge that they exist based primarily on what people say and do. GPT-style models can't do anything, but they can say. And for better or worse a whole lot of ordinary, everyday speech doesn't have a whole lot of thinking, comprehension or imagination to it. It's just people saying what other people seem to be saying in order to try and stay with the herd. GPT-3 is scarily close to matching that.

So, they're getting pretty close to matching us in mindlessness and stupidity. Call me when they can match us in our highs, not in our lows.

Yep, pretty much :)

It's pretty fascinating that nobody (AFAIK) really predicted that, except perhaps Isaac Asimov. Sci-fi normally presents AI as either superintelligent or non-existent (just basic voice commands). I don't think I've seen one which presents it as an AI ecology of idiocracy.