> How do you verify if what ChatGPT returns is actually correct?

A bit ironically, if it is regarding a fact, I perform a search on DDG or google and find an authoritative website or paper to corroborate it. So it still feels like search outside of an LLM is an important tool. If the result is code I can test it myself.

There needs to be some lexicon for this kind of thing. Theres a parallel with NP problems, some things are hard to figure out but easy to verify. Others hard to do both.

Bing is a major step forward where it cites what sources it uses, when it uses sources.

It works, too. I asked it for the history of a term, and it gave a very plausible-sounding answer citing three different links. All of them were to the same page that didn't say what Bing said it did. A rare miss, but at least I could check its work.

What I do is I don't ask it to generate results, but to give me a program that does the processing, then have it give me a test suite where I can 'trivially' (usually) verify the program on synthetic data (which, of course, I also have ChatGPT create for me). It usually even includes good edge cases.

I've had a few times where I didn't like the library that it used to come to the results, I just told it 'use library xyz' and it would rewrite the whole thing using idioms native to that library. It's amazing. It's eliminated like 75% of the drudgery that I've come to hate so much about programming the last years.