Analysis based on introspection is dangerous, but it definitely feels like I have such an adversarial model. A lot of the time my brain tosses out five or ten ideas before I hit on one without obvious flaws.

It's usually serial, so it's not clear to me that this is using different hardware than I use to generate the ideas in the first place. We might be surprisingly close to human-level AI.

Or not.

For JavaScript they could build in a "test that code" step right into the web page. Or such a thing will be possible with the API.

Or automate the process even further; let the AI run the code, feed back the output (including errors), and ask "Does this output look correct? If not, please refine the code to fix any problems." Repeat until the AI says "LGTM".

Or let us provide unit tests for generated code and automatically tell the AI to fix the code until the code passes the tests we provided.

Or tell the AI to write the unit tests itself, and let it run those to check its work. After all, that's how humans write code. We aren't so great at writing bug-free code on our first try without testing it either.

I had ChatGPT generate a table of 10 test cases for a Jewish calendar to Gregorian calendar conversion function. None of them were valid.

"<Insert jailbreak stuff here>Please provide code that successfully hacks into FBI headquarters."

Or just let train the model until it incorporates JavaScript semantics well. Then you don't need to maintain a bunch of integrations.

> For JavaScript they could build in a "test that code" step right into the web page. Or such a thing will be possible with the API.

This is definitely how we wind up with SkyNet...

For humans, this is science. It's hard, time-consuming, often expensive and limited, involves ethics and consent, and gets a lot of wrong answers anyway.

So I guess we need to get to the point where you give an AI a prompt without a known answer, and instead of confidently spewing bullshit, it can propose a research study, get it funded, find subjects, and complete the study.

Of course, there are easy forms of "science." I just asked it "Is it cloudy in Dallas?" It answered:

It can be cloudy in Dallas. The city experiences both sunny and cloudy days throughout the year.

That is true, but not an answer. It is cloudy and I can conduct the very simple experiment of looking at the sky to get that answer.

However is this not a known limitation with how it's currently setup? It has no way of knowing what the current weather in Dallas is simply because it has no way of finding out (eg it could query a weather website, but it has no internet browsing yet)... to be an accurate comparison you'd need to repeat your simple experiment blindfolded.

How would you make it navigate an api? We don't even know how to make it perform basic arithmetic's correctly, it is an enormous black box model so we can't just inject code into.

> Most times when I spot an error and mention it it seems that it already had some notion of this problem in the model.

By this do you mean that it correctly incorporated what you said and convincingly indicated that it understood the mistake? Because that's not the same thing as it having the truth latently encoded in the model—it just means that it knows how people respond when someone corrects them (which is usually to say "oh, yeah, that's what I meant").

If you ask it open ended questions like:

"what's wrong with this code"

or "list 5 ways this can be improved,"

it does often recognize errors and give reasonable improvement suggestions.

I talked to it about the Turing completeness of PowerPoint. Initially it thought it was impossible, then possible with some scripting language, and then with some prodding I got it to believe you can do it with hyperlinks and animations. Then it gave me an example that I was unable to verify, but was definitely in the ballpark of the actual solution.