News for hackers. Stuff that matters.
Re:Copilot regurgitating Quake code, including sweary comments by thinkingemote on Friday July 02, 2021 @08:07AM (#27710393)
From the GPLv2 licensed code:

https://github.com/id-Software/Quake-III-Arena/blob/master/c...

copilot repeats it word for word almost, including comments, and adds an MIT like license up the top

Re:Copilot regurgitating Quake code, including sweary comments by Thomashuet on Friday July 02, 2021 @09:12AM (#27711057)
Actually the indentation of the first comment and the lack of preprocessor show it's not copied from this code directly but from Wikipedia (https://en.wikipedia.org/wiki/Fast_inverse_square_root#Overv...) So It could be that the Quake source code is not part of the training set but the Wikipedia version is.
Re:Copilot regurgitating Quake code, including sweary comments by SamBam on Friday July 02, 2021 @09:55AM (#27711562)
While I strongly doubt they would use Wikipedia as a training set, has anyone done a search of GitHub code to see if other projects have copied-and-pasted that function from Wikipedia into their more-permissive codebases?
Re:Copilot regurgitating Quake code, including sweary comments by bootlooped on Friday July 02, 2021 @11:45AM (#27712992)
Almost 2000 results for one of the comment lines. I'm not going to read through those or check the licenses, but I think it's safe to say that block of code exists in many GitHub code bases, and it's likely many of those have permissive licenses. Given how famous it is (for a block of code) it's not unexpected.

https://github.com/search?q=%22evil+floating+point+bit+level...

A question that popped into my head is: if the machine sees the same exact block of code hundreds of times, does that suggest to it that it's more acceptable to regurgitate the entire thing verbatim? Not that this incident is totally 100% ok, but if it was doing this with code that existed in only a single repo that would be much more concerning.

Re:Copilot regurgitating Quake code, including sweary comments by Animats on Friday July 02, 2021 @01:53PM (#27714607)
if the machine sees the same exact block of code hundreds of times, does that suggest to it that it's more acceptable to regurgitate the entire thing verbatim?

From a copyright standpoint, quite possibly. This is called the "Scènes à faire" doctrine. If there are some things that have to be there in a roughly standard form to do a standard job, that applies.

[1] https://en.wikipedia.org/wiki/Sc%C3%A8nes_%C3%A0_faire

Re:Copilot regurgitating Quake code, including sweary comments by nextaccountic on Saturday July 03, 2021 @02:31AM (#27719804)
This would need to first be tested in court; apparently Microsoft is happy in generating thousands (or millions) of violations, knowing most programmers don't enforce their copyright.
Re:Copilot regurgitating Quake code, including sweary comments by ajayyy on Friday July 02, 2021 @10:42AM (#27712136)
It is probably based off GPT-3 with a layer on top trained for programming specifically, like what is done with AI dungeon.
Re:Copilot regurgitating Quake code, including sweary comments by an_opabinia on Friday July 02, 2021 @02:09PM (#27714778)
Wait until people on the toxic orange site find out what has happened to AI Dungeon.
Re:Copilot regurgitating Quake code, including sweary comments by SamBam on Friday July 02, 2021 @02:10PM (#27714793)
I'm out of the loop.
Re:Copilot regurgitating Quake code, including sweary comments by grawprog on Friday July 02, 2021 @02:29PM (#27715019)
Re:Copilot regurgitating Quake code, including sweary comments by zxzax on Friday July 02, 2021 @04:50PM (#27716691)
I don't get it, that seems like standard fare for an R-rated movie? And then it seems like some complained because they decided to start editing it down to a PG-13 movie?
Re:Copilot regurgitating Quake code, including sweary comments by grawprog on Friday July 02, 2021 @05:06PM (#27716872)
Essentially, from my understanding, there was a data leak they never commented on, they instituted a poorly made content filter without saying anything. The filter frequently has false positives and negatives, someone discovered they trained the game using content the filter was designed to block, meaning the ai itself would frequently output filter triggering stuff, more people found out their private unpublished stories were being read by third parties after a job ad and the stories were posted on 4Chan, people recognized stories they wrote that had triggered the filter that were posted, and then they started instituting no warning bans.

I might have missed something, but that's the gist of it.

Re:Copilot regurgitating Quake code, including sweary comments by Filligree on Friday July 02, 2021 @08:48PM (#27718454)
Also, before and while all this was going on, the quality of the AI's output has been steadily dropping to the point where NovelAI.net now generates what's in many ways better writing.

That's GPT-J-6B, to be clear. A 6-billion-parameter model is producing better output than a 300 billion parameter model, because of what I can only assume to be sheer incompetence on AI Dungeon's part. I've also used the raw GPT-3 API, and it does better at writing than either. In other words: Doing nothing would have been better than whatever they've been doing.

Re:Copilot regurgitating Quake code, including sweary comments by armatav on Friday July 02, 2021 @02:26PM (#27714983)
It’s pre-trained, partially, on Wikipedia. GPT-2 did this sort of thing all the time: native to the architecture to surface examples from the fine-tuning training set by default.
Re:Copilot regurgitating Quake code, including sweary comments by edgyquant on Friday July 02, 2021 @10:55AM (#27712316)
It’s GPT though and the GPT models were trained on data from Wikipedia
Re:Copilot regurgitating Quake code, including sweary comments by rmorey on Friday July 02, 2021 @09:28AM (#27711230)
This exact code is all over github, >1k hits

https://github.com/search?q=%22i++%3D+%2A+%28+long+%2A+%29+%...

Re:Copilot regurgitating Quake code, including sweary comments by nextaccountic on Saturday July 03, 2021 @02:25AM (#27719783)
Then they all copied from the same source, and what's more, they are all derivative of a GPL work (and thus they should be GPLed themselves)
Re:Copilot regurgitating Quake code, including sweary comments by ajklsdhfniuwehf on Friday July 02, 2021 @02:25PM (#27714972)
that will make a great defense at a copyright court.

"your honor, i would like to plead not guilty, on the basis that i just robbed that bank because i saw that everyone was robbing banks on the next city"

...on the other hand, that was the exact defense tried for the capitol rioters. So i don't know anything anymore.

Re:Copilot regurgitating Quake code, including sweary comments by arksingrad on Friday July 02, 2021 @10:32AM (#27712017)
I guess this confirms John Carmack to be an AI
Re:Copilot regurgitating Quake code, including sweary comments by OskarS on Friday July 02, 2021 @11:39AM (#27712905)
Apparently Carmack was not the original author, the origin I believe is SGI somewhere in the deep dark 90s.
Re:Copilot regurgitating Quake code, including sweary comments by Haga on Friday July 02, 2021 @02:44PM (#27715216)
Was a optimization for a fluid simulation originally..
Re:Copilot regurgitating Quake code, including sweary comments by Fordec on Friday July 02, 2021 @03:06PM (#27715486)
I get why some people were saying it made them a better programmer. Of course it did, it's copy-pasting Carmack code.
Re:Copilot regurgitating Quake code, including sweary comments by nojito on Friday July 02, 2021 @09:14AM (#27711070)
It's up to the end user to accept the suggestions.
Re:Copilot regurgitating Quake code, including sweary comments by croes on Friday July 02, 2021 @09:26AM (#27711212)
Good luck checking every code line for license violations
Re:Copilot regurgitating Quake code, including sweary comments by dr_kiszonka on Friday July 02, 2021 @12:32PM (#27713688)
There will be a VSCode extension for that.
Re:Copilot regurgitating Quake code, including sweary comments by TheDong on Friday July 02, 2021 @05:13PM (#27716944)
It's impossible to automate checking for code license violations.

If you and I write the exact same 10 lines of code, we both have independent and valid copyrights to it. Unlike patents, independent derivation of the same code _is_ a defense for copyright.

If I write 10 lines of code, publish it as GPL (but don't sign a CLA / am not assigning it to an employer), and then re-use it in an MIT codebase, I can do that because I retained copyright, and as the copyright holder I can offer the code under multiple incompatible licenses.

There's no way for a machine to detect independent derivation vs copying, no way for the machine to know who the original copyright holder was in all cases, and whether I have permission from them to use it under another license (i.e. if I email the copyright holder and they say 'yeah, sure, use it under non-gpl', it suddenly becomes legal again)...

It's not a problem computers can solve 100% correctly.

Re:Copilot regurgitating Quake code, including sweary comments by dr_kiszonka on Friday July 02, 2021 @09:03PM (#27718506)
I should have added /s to highlight that I was being sarcastic. Sorry.
Re:Copilot regurgitating Quake code, including sweary comments by croes on Friday July 02, 2021 @12:50PM (#27713914)
Same trust issue
Re:Copilot regurgitating Quake code, including sweary comments by atatatat on Friday July 02, 2021 @01:38PM (#27714432)
It's people for your lawyers to blame, all the way down!

/s

Re:Copilot regurgitating Quake code, including sweary comments by croes on Friday July 02, 2021 @03:27PM (#27715751)
It's the same problem s with self driving cars, you gets sued. The company that provides the service/car or the the programmer/driver? I think the latter.
Re:Copilot regurgitating Quake code, including sweary comments by duckmysick on Friday July 02, 2021 @02:02PM (#27714686)
SaaS idea: code linter, but for licenses.
Re:Copilot regurgitating Quake code, including sweary comments by SahAssar on Friday July 02, 2021 @02:27PM (#27714998)
That's one of blackduck's offerings: https://www.synopsys.com/software-integrity/open-source-soft...

At a previous job we had a audit from them, it seemed to not be too accurate but probably good enough for companies to cover their asses legally.

Re:Copilot regurgitating Quake code, including sweary comments by adrianN on Friday July 02, 2021 @02:21PM (#27714926)
Extend Fossology: https://www.fossology.org/
Re:Copilot regurgitating Quake code, including sweary comments by freshhawk on Friday July 02, 2021 @09:20AM (#27711148)
And it's up to the end user to evaluate the tool that makes the suggestions.
Re:Copilot regurgitating Quake code, including sweary comments by Spivak on Friday July 02, 2021 @10:48PM (#27718976)
This is true but doesn't change the problem that copilot itself is potentially distributing unlicensed copyrighted material. This isn't necessarily a problem for you as a developer though.
Re:Copilot regurgitating Quake code, including sweary comments by flatiron on Friday July 02, 2021 @10:29AM (#27711967)
As someone who does code reviews the thought the developer didn’t code the code submitted to be merged never would cross my mind.
Re:Copilot regurgitating Quake code, including sweary comments by user-the-name on Friday July 02, 2021 @10:15AM (#27711788)
And it is completely impossible for the user to do so.

So, the tool is worthless if you want to use it legally.

Re:Copilot regurgitating Quake code, including sweary comments by nojito on Friday July 02, 2021 @10:28AM (#27711956)
Doubtful.

You can be almost certain it’s being widely used or will be widely used shortly.

The conversations around copilot are eerily similar to the conversations around the first autocomplete tools

Re:Copilot regurgitating Quake code, including sweary comments by gnulinux on Friday July 02, 2021 @11:48AM (#27713042)
It's more like a writer using an autocomplete tool to write the first chapter to their novel.
Re:Copilot regurgitating Quake code, including sweary comments by caconym_ on Friday July 02, 2021 @02:03PM (#27714697)
As someone who gets paid to write code (nominally) and has also written a few novels, I don't agree with this characterization. From what I've seen of Copilot, it's more like having a text editor generate your next sentence or paragraph^[1]. The idea (as I see it) is that you might use it to generate some prose "boilerplate", e.g. environmental descriptions, and hack up the results until you're satisfied.

It's content generation at a fragmentary level where each "copied" chunk does not form a substantive whole in the greater body of the new work. Even if you were training it on other authors' works rather than just your own, as long as it wasn't copying distinctive sentences wholesale, I think there's a strong argument for it falling under fair use--if it's even detectable.

On the other hand, if it regurgitated somebody else's paragraph wholesale, I don't think that would be fair use. Somewhere in-between is where it gets fuzzy, and really interesting; it's also where internet commenters seem to prefer flipping over the board and storming out convinced they're right to exploring the issues with a curious and impartial mind. I see way too much unreasoned outrage and hyperbolic misrepresentation of the Copilot tool in these threads, and it's honestly kind of embarrassing.

As far as this analogy goes, it's worth noting that the structure of a computer program doesn't map onto the structure of a piece of fiction (or any work of prose) in a straightforward way. Since so much of code is boilerplate, I would (speculatively, in the copyright law sense) actually give more leeway to Copilot in terms of absolute length of copied chunks than I would for a prose autocompleter. For instance, X program may be licensed under the GPL, but that doesn't mean X's copyright holder(s) can sue somebody else because their program happened to have an identical expression of some RPC boilerplate or whatever. It would be like me suing another author because their work included some of the same words that mine did.

^[1] At least one tool like this (using GPT-3) has been posted on HN. At this point in time I wouldn't use it, but I have to admit that it was sort of cool.

Re:Copilot regurgitating Quake code, including sweary comments by Filligree on Friday July 02, 2021 @08:52PM (#27718476)
> ^[1] At least one tool like this (using GPT-3) has been posted on HN. At this point in time I wouldn't use it, but I have to admit that it was sort of cool.

Have a poke at novelai.net if you get a chance.

It's... not very smart. It's pretty decent at wordcrafting, though, and as an amateur writer I find it invaluable for busting writer's block. Probably if you spend all day writing fiction you'll find ways around that, but for me the solution has become "Ask the AI to try".

It'll either produce a reasonable continuation, or something I can look at and see why it's wrong. Either is better than a blank page.

Re:Copilot regurgitating Quake code, including sweary comments by caconym_ on Sunday July 04, 2021 @02:43AM (#27727722)
In case you're interested, this is the post I was thinking about: https://news.ycombinator.com/item?id=27032828

The application itself is called "Sudowrite". I guess there are probably a bunch of them at this point.

Re:Copilot regurgitating Quake code, including sweary comments by user-the-name on Friday July 02, 2021 @10:58AM (#27712350)
That does not seem like a response to what I just said?

I said that it is impossible for the user to check that the code copilot gives is OK, license-wise, and therefore, they can not be sure that it is legally OK to include in any project.