OK, have it do something novel that's not a mashup of its training data or the prompts you feed it.

> We need a way to tell the difference between something a human wrote by "mashing up" it's "training data" and a something a computer wrote doing the same thing.

I don't think people really even think that way, they would be confused by the question. This is more of a niche subject of AI research and scifi, these concepts are not understood at all by normal people, normal people say things like:

Why do we need to tell the difference? We know if something is being generated by a machine unless you hide that fact to cheat on your homework or something. A machine is made by humans, using some wires or a bunch of if then else programming or something, its always going to be different to a human of course, humans are sentient, machines are not.

If you tell them about the control problem they just tell you to pull the plug if it does start acting weird. This stuff is very challenging to communicate to people who aren't already reading lesswrong.

1. No, it hasn't.

2. The Turing Test is actually not a great test of machine intelligence.

You could call this magic a "mashup of its training data," but I'd say it's more like genuine learning.

I’m very good at creative brainstorming—but I have to say, this tool is extremely good at creative, insightful brainstorming. Just for a recent example, I asked it anticipate backlash to ChatGPT—to which it gave ideas I’d not considered—how could that be in it’s training data?

It’s ok to say “only conscious beings can be truly intelligent and machines can never be.” But apart from this perspective , I don’t understand the reluctance to declare this machine to be generally intelligent. It is an amazing accomplishment and I think should be recognized as such. Besides, the exponential growth in general intelligence is likely to continue for quite some time.

This is kind if like saying "I typed a question I know the answer to into a search engine and it showed me webpages in the results where the answer was something I hadn't considered before"

Obviously it's in the training model because someone else aside from you did think of it before.

Or maybe it includes backlash against ML models, and ChatGPT "knows" that it is one of those.

There is no magic here.