Whether or not the result is a license violation is tricky legal question. As always, IANAL.
It seems to me an important question is, "is this like a human who learns from examples, or is this really a derivative work in the copyright sense?".I'm not sure how to answer that. I'm not a lawyer. I don't know if many lawyers can answer that question either!
https://scholarship.law.cornell.edu/facpub/1481/
Copilot isn't human and therefore what it does isn't a "work".
The usual issues still apply to users of Copilot - unwitting violations of license terms of the code it was trained on (like non-attribution) are still violations.
I don't have photographic memory, so I largely don't memorize code. I learn general techniques, and memorize simple facts such as APIs. I can memorize some short snippets of code, but these probably aren't enough to be copyrightable anyway.
> The type of model they use isn't retrieving
How do we know? It think it's very likely that it is largely just retrieving code that it memoized, and doing minor adjustment to make the retrieved pieces fit the context. That wouldn't differ much from finding code that matches the problem (whether on SO or Github), copy pasting the interesting bits, and fixing it until it satisfies the constraints of the surrounding code. It's impressive that AI can do that, but it doesn't sound like it's producing code.
I think the alternative to retrieving would actually require a higher level understanding of the world, and the ability to reason from first principles; that would be much closer to AGI.
For example, if I want to implement a linked list, I'm not going to retrieve an implementation from memory (although given that linked lists are so simple, I probably could). I know what a linked list is and how it works, and therefore I can produce working code from scratch.. for any programming language, even ones for which no prior implementations exist. I doubt co-pilot has anything remotely as advanced as this ability. No, it fully reliant on just retrieving and reshaping a pieces of memoized code; it needs a large corpus of code to memoize before it can do anything at all.
I don't need a large corpus of examples to copy, because I use my ability to reason in conjunction with some memoized general techniques and common APIs in order to produce original code.
If it's spitting out verbatim code 0.1% of the time, surely it's spitting out copied code where only trivial things are different at a much higher rate.
Trivial things meaning swapped order where order isn't important, variable/function names, equivalent ops like +=1 vs ++, etc.
Surely it's laundering some GPL code, for example, and effectively removing the license in a way that sounds fishy.