It depends, it happened for NN to spit out exact strings it had as input. I assume if the input for some particular stuff is low it will just spit it out, but if I am wrong and OpenAI, Microsoft or others have a mathematical proof that this will never happen I want to see it.

Yes we have seen a case of an old common and popular square root function that are returned verbatim by GitHub Copilot. Perhaps a case of overfitting since the code was present many many times in the training dataset, and even has a Wikipedia page.

I haven’t seen more obvious examples since GitHub implemented a feature to prevent this from happening. I probably miss some tweets, but I assume it’s rare.

Carmack’s fast inverse square is not subject to copyright in the first place.

True. I think the comments and the coding style were from famous implementation from the 90s, which could be a copyright issue. The algorithm itself can by copied.

In the case of ChatGPT, you can be almost sure that if a non-trivial amount of code works in production then it's an unlicensed paraphrase of some human written code. It treats code (and everything else) like prose, hence the atrocious results in the original story.

That does not mean other coding-optimized AI models won't be able to do much more, symbolic reasoning about the data, maintaining consistent variable and function names, understand libraries, respecting language constraints and invariants etc.

There’s no such thing as an “unlicensed paraphrase” of small snippets of utilitarian software, at least in US copyright law.

You might have moral qualms with this but you won’t find much of any support from the court system.

It's clearly a derived work, since the apparatus that has produced the work (the AI model) can only function when supplied with copyrighted works - in fact it cannot exists without copious amounts of such works. If there is even a passing semblance between a source work and the output, such as, for example, similar code structure and different variable names, then you are legally fucked.

I'm sure courts will clear this mess up really soon, and I'm betting money the rulings won't subscribe to the "it's mine now" mantra of the AI crowd.

Do you actually want to bet money on this? I’m willing to play!

Is there any planned court case about the copyright fair use question ?

I think ChatGPT switch to OpenAI codex when it writes code. It’s the same model that GitHub Copilot uses.