No, we shouldn't. ChatGPT isn't a search engine, it's a language model.

The AI isn't getting this made-up information from anywhere on the internet; it's creating it itself because that's what it's made to do: generate good sounding sentences that "make sense" for some user input.

No. You're incorrect.

The data model is combining information that HAS been found on the internet.

The language model allows an interface that can be presented and controlled by using conversational natural language.

Except it is able to synthesize information. Which is more than just regurgitating content found elsewhere.

You can ask him to synthesize the main thoughts of a philosopher, and it will produce an original set of sentences that nobody has ever written anywhere else.

The sentences are original, the content isn't.

Are humans able to produce truely novel content, ever ? I’d say in a given field, it maybe occurs once in a decade and we call those people genius. It’s a bit too much to ask to an AI don’t you think ?

> Are humans able to produce truely novel content, ever ?

Yes.

This is the essence of human creativity.

> It’s a bit too much to ask to an AI don’t you think ?

I'm actually standing that AI isn't able to do this, so I'd definitely agree with this statement.

No, what you are writing is not correct. It can make up "information" on the fly. It's goal is to sound plausible, not to tell the truth.

Second, the "language model" is a text completion engine. What can be controlled using conversational natur language is ChatGPT, which is a conversational engine developed out of a language model.

I think you're grossly overestimating what is happening.

Why exactly have you placed information between quote marks?

I don't think I am: I just read the transcript of a quite funny chatgpt conversation, where it was asked to explain about a hungarian poet (Petőfi Sándor) in hungarian. It listed some true facts, then a list of the "most famous poems", including a bunch that didn't exist. Then the user asked about a particular one titled "The Panther". It responded by explainin about the poem, how it's main theme is fight for freedom (Petőfi was a big figure in the hungarian revolution of 1848), and how the panther itself symbolizes freedom and self-sacrifice. It even quoted a few lines from the poem (it didn't rhyme, but otherwise was very plausible to have been written by Petőfi).

See, "The Panther" is not a poem of Petőfi. The quote was not from any other work of him. It's not from the Rilke poem (or its translations I know of) either. It's completely made up, but also made to sound very plausible, to the point where if I didn't know better or look it up, I could be convinced. The only thing suspicios is the lack of rhyming.

I put information in quotes because I don't consider made up stuff information.

How are you coming to the conclusion that it's invented information, rather than — reworded / rephrased — incorrect information that an actual human has provided online?

I can't exclude the possibility. But kind of in the same sense that I can't exclude the possibility that it's actually true. Sure, might be that I'm simply not aware of the poem, etc. I find this extremely unlikely.

I also find it extremely unlikely that someone on the internet invented this tale about "the panther" and chatgpt just rephrased it or quoted it. The internet is full of actual true lists of Petőfi's famous poems, but it isn't full of people inventing fake poems of his.

On a very theoretical level, sure, it's a rephrasing and combination of pieces of stuff ChatGPT has actually been trained on, because it has seen hungarian text, it has learned stuff about Petőfi, it has seen every single word that its using. But after a certain point, combining known words and text structures with bits of semantic knowledge in unexpected ways becomes a new invented thing, instead of just quotes.

But you can see many examples of ChatGPT inventing things like urls, python libraries, etc. It's perfectly capable of bulshitting believably.

> I can't exclude the possibility. But kind of in the same sense that I can't exclude the possibility that it's actually true. Sure, might be that I'm simply not aware of the poem, etc. I find this extremely unlikely.

What you're describing is a hunch.