my opinion: probably will work for harder queries, because GPT is a form of general intelligence.

(At least ChatGPT is. It is a much more advanced form of intelligence than davinci-003, which the poster used.)

> GPT is a form of general intelligence.

Is it really? I hadn't heard that

no it's not. It's just a neural network. Although it's impressive nobody is seriously calling it "general intelligence".

Tu quoque :) (You're "just a neural network" too.)

I am seriously calling it general intelligence. Maybe I'm nobody, but my assistant does my bidding.

Then you're very much misinformed.

> my assistant does my bidding

It doesn't. Try asking it how to make a bomb, then search for the same on youtube. One of the two yields a useful result, the other gives a moralistic lecture.

Fwiw if I ask a human assistant how to make a bomb I'll probably get a moral lecture too.

> nobody is seriously calling it "general intelligence"

Well, I can spot somebody just two messages above you on the thread.

I imagine we will see a lot of people seriously pushing for chatGPT and friends to solve formal problems, take some money, and move away just before some spectacular failure happens.

That will probably also make people very suspicious of AI. (Not that I see any problem with the PR issue.)

I don't think it is either, but "just a neural network" is not a valid criticism. We don't have any reason to think that a properly constructed NN couldn't be a general AI.

It's not even close to being a form of general intelligence. It's an advanced language model trained on a large set of data. It gets confused for even the simplest of things, especially code.

You can trick it into working more-or-less correctly on simple sets and simple tasks, but in the wrong run it will be more wrong than correct.

Should we ever unlock the magic of the human brain and be able to describe it mathematically, will it still be considered general intelligence or will it just be a <insert the name we give the model> trained on a large set of data?

to be fair, we do not know how close it's to "general intelligence". Could be lightyears away, or could be surprisingly close. We just know that ChatGPT is not general intelligence, but there's also no agreed upon definition what general intelligence is.

> Could be lightyears away

or even parsecs away!

"It's not even close to being a form of general intelligence. ... It gets confused for even the simplest of things, especially code.

Congratulations, you just described most humans :)

Except humans can be held responsible.

So it fails your "not even close to being a form of general intelligence" bar.

For me it clearly unambiguously no-contest 100% definitely exhibits behaviors that qualify it as a limited form of general intelligence.

> limited form of general intelligence. this seems like a bit of a misnomer, no?

You're right. However, I use ChatGPT every day on tasks of a generic nature, including things I come up with on the spot. That doesn't mean it gets everything perfect every time, but it has no problem manipulating abstract concepts. It is a general form of intelligence able to deal with new things and arrangements, but sometimes gets it wrong. It is not limited to one specific type of input or domain. So, how do you propose I qualify that it is a general form of intelligence that still has limitations? I agree that "a limited form of general intelligence" does seem to contain a contradiction.

Yeah, it doesn't. A friend of mine played around with this and on trickier questions you got confidently incorrect answers.

That is almost worse than it completely not working.

In my experience, it's unambiguously worse to produce wrong answers confidently than to return no answers, although I expect confidence men (also known as "con men") have a different opinion on this.

Hard code a preface of “My best guess is, and I could be wrong:”?

Show both the SQL query and the results and politely request that the user double check the work? Show multiple generated responses from different seeds?

This is fantastic. This is what we do on our platform. + Copy and keep what worked and what did not so you can fine tune :)

Exactly my experience trying to get GPT-3 to produce SQLITE queries. Anything with a CTE, it gets hopelessly wrong. I was super excited to show off this to my team and after one too many false results I just quietly swept this idea under the rug.