This seems like a key line.
I can get my browser to output GPL code with a carefully crafted Google search query, and there might even be unlicensed use of GPL code in my browser itself. But as I understand it, as long as I don’t copy GPL code, I should be fine.
In this case they’ve disabled the GPL filter, and gotten GPL’d outputs. Fair enough. The question though: when the filter is on, can I trust it? Not sure!
Secondly, if you take this seriously you should already being doing some kind of static analysis to detect it because what if one of your employees copy/pastes some GPL code from somewhere?
It's practically a non issue that people are blowing out of proportion.
I don't think it is. If it is legal to input copyrighted stuff into an LLM and have the LLM magically remove the copyright, then the very concept of copyright is dead. LLMs are then a copyright-laundering machine.
Worse: if it is considered "fair use", then I don't think that I could even sell my copyrighted material with a contract that says "you are not allowed to feed it to your copyright-laundering machine". If it is fair use, anyone can do it, right?
1. The LLM might output GPL (or other restricted) code if you turn off the filters.
This one I see as a non issue.
2. The LLM benefited from GPL code and various copyrighted content in its training.
This is more complicated and for the courts to decide. My personal opinion is that as long as you as the end user are not publishing the output copyrighted content it shouldn't matter, and if you are then the owner of that copyright should pursue justice through the existing processes.
But yeah, ignoring the issues others have is a good way of living in a world with no issues at all.
Althought at that point, a well-intentioned human seeing the repo would similarly be fooled.