This would be way more fun

Well, if they still have access to ChatGPT, they can ask that of it as well. It may or may not give a valid answer, just as it does with code.

A big if. A lot of ChatGPT discussions seem to take for granted that it'll always be available/free/priced low enough that ~everyone has access to it. Seems more likely that at some point OpenAI will close it up and put it back behind an API.

You can use text-davinci-003 from the API now and it works better for many things.

The company itself could have access to the AI for the interview, since idea is lowering the playing field.

But even more, it seems likely someone is going to be willing to supply LLM access at not that much more than the cost of computation.

I know OpenAI's business is selling access to their stuff at a premium but since current AI is not much more than brute-forcing of massive public data, it doesn't seem like this premium will be justifiable for long.

And you can give every candidate a new version of it.

I love it!

Particularly, ChatGPT can give apparently correct solutions that are wrong in subtle ways.

Also reading, understanding, reasoning about, and fixing code other's wrote is way closer to on-the-job performance.

Yep, the future of SWE is debugging AI.

This is already the present for translators. They often are called to proofread machine translated text. And this is often more difficult than rewriting the whole text.