How likely that situation is is disputable. People have cherry picked some cases where by specifying a very particular comment you can get it to (unreliably) reproduce well-known pieces of code, sure. But that is neither the intended use of the product nor the actual way that a single person using the product really employs it, and that really, really matters. Judges are not automatons, and the fact that when and if this goes to court the developers will be able to get up and honestly say "this is a tool developed to help developers create new code that is not merely recreating the functionality of the code we trained the tool on, and all our users use it to create entirely new things" is going to matter when they argue that it is fair use/transformative work.

I do completely understand that a lot of people disagree with me morally, and think that extracting insights from scraping the public web should be illegal. You're free to have that opinion, but I'd recommend you start lobbying your congressman to change the law, because though I'm not a lawyer I hang out with a few who do copyright stuff, and I don't think the law as it stands is on your side. That said, who can say, maybe this will end up bubbling up through many layers of appeals and end up at the Supreme Court someday, this stuff is all certainly wildly outside the bounds of what anyone writing the copyright laws was thinking about back in the 70s (which I think was the latest significant iteration?) so it's fair to say it's a complete gray area.

>But that is neither the intended use of the product nor the actual way that a single person using the product really employs it, and that really, really matters

And yet, someone has done so, and found that to be the case. Therefore, it falls under "normal use".

So make up your mind. Is software engineering only about assembling working programs, or assembling working programs + navigating license minutiae?

One of these, Copilot has a place in. The other it does not.

You may hope that a judge considers a really weird and cherry-picked weird edge case to be "normal use", but opposing counsel will argue the opposite, and have a lot of evidence if they actually look at real use.

"I can use a gun to shoot someone" doesn't make guns illegal, even if people do so with some regularity. "I shot someone to make the point that guns can kill people" is worth even less in the eye of the law, and that is literally what you're pointing to here.

And I don't know what you're even getting at with the "assembling working programs" vs "navigating license minutae" stuff. Copilot helps the former, and you're apparently trying to argue that it should be banned because some people are feeling angsty about the latter?