We used the repo-level license information from github to filter to free non-copyleft licenses, and the license files are actually stored next to the extracted data corpus (see https://github.com/github/CodeSearchNet#Licenses).
It seems irresponsible for Github to continue distributing infringing code when it clearly has the tools to prevent that happening. Youtube has fingerprinting for infringing music and video. Github should be doing the same with source code.
If you're saying they are the entity in the best position to identify infringement, I would agree, but what tools does Github already have in place to do that?
By failing to do so, Github really seems to be encouraging copyright infringement.
If Github were somehow helping copyright infringers, that would be encouragement. What you're describing is additional effort on their part to actively discourage copyright infringement, which is not the same thing.
What are you talking about?
From the paper, it's not really clear how the resulting binary or intermediate representation fits into the picture? Wouldn't a robust model have to exploit features of the underlying computer architecture as well?
Besides the possible applications mentioned such as recommendation systems and code generation. I am wondering about automatic code annotation and documentation. With the eventual goal of creating a "code tutor" that would assist students in real-time as they type. I think what's interesting about this is that the most talented CS grads tend to have a high correlation with deep understanding not only of the history of digital logic design, but also math, physics etc.
Fantastic work ;)
Thanks for your questions! We have thought of many heuristics but we didn't want to constrain the dataset release on some heuristic that we picked, possibly ruining the dataset. Participants in the challenge should feel free to apply additional filters as they see fit. For example, this [1] work could be useful as a filtering method.
Unfortunately, we do not have the budget to provide any compute resources to help with running the models at this time. Note that any techniques developed with this dataset will be owned by those who develop them and it's up to them how/if they will make them available/open-source.
All code is from public repositories. We only used publicly available APIs to obtain the data, so that others can reproduce the results / build on top of the data extraction pipeline we built: https://github.com/github/CodeSearchNet/tree/master/function...
It's also easy to test without access to private repos, by just splitting some repos off as a separate test set, so should hopefully be something that other people can also make progress on!
Do you realize how rude and disrespectful that tone is?
There is a civil way to air one's views; one that we use naturally in face-to-face conversations. Yet once it's from behind a computer screen, we somehow seem to think it's okay to lash out in a barrage of hurtful language...
C’mon, cut these folks some slack — GitHub is huge, and one annoying adjacent feature doesn’t give you the right to attack these folks like this.
> Just get the stupid dedupping done, for God's sake.
> You have no excuse to so badly that a teenager can beat you by simple handcrafted rules.
> First learn and do the basics and then come here to talk about your need for ML.
> No one needs to suffer like this.
Let’s see, the OP belittles the original authors for something they’re not responsible for, implies that they’re incompetent, tells them to go away until their problem is fixed, and, oh, for god measure: invokes a deity.
This is not “straight forward feedback” to me.
How about we just cut all the insults and shift the tone a bit:
> GitHub has a really poor search experience compared with most modern web services. Search doesn't need fancy ML or DL, and dedupping alone would help a ton. A simple TF-IDF similarity metric would make GitHub’s search 10,000% better. You even know forks, names of files, folders, number of stars.
> PS: Sorry, I had to get this out. I've spent better part of my productivity in past two years shifting through page number 50 and 100 of undedupped results that GitHub returns and it’s been extremely frustrating.
See how this doesn’t blame the authors for something that isn’t their fault? Doesn’t imply they’re stupid or incompetent for some other feature they have nothing to do with? Doesn’t tell them to go away? And a bonus: It presents some approaches to solve a problem the author is frustrated by.
My experience ranting on the internet is that personal attacks don’t lead to feature improvements.
I think the pointed out asymmetry was spot on.
Something like a robots.txt for github projects. Not that anybody would really care, only to make my intent clear that I don't support this sort of mass data gathering nonsense.
I consider this mass data gathering use for machine learning an edge case which should not harm the 'legitimate' users of the repository.
While we present the task of information retrieval as one possible use case of this dataset, we know there could be other practical applications of this data (i.e. code summarization). While we went through great lengths to pre-process the data for the community, the data is still messy and often you will find that there might not be high-quality comments that are aligned with a code snippet we parsed. However, we believe this is part of the excitement of the dataset it poses challenges that machine learning practitioners will have to address.
Code is very different than natural language with regard to its structure and syntactic rules and may benefit from different approaches relative to natural language processing. Our baseline models and benchmarks mostly treat code as natural language, however we are aware that there could be an opportunity to innovate on this front. If anyone creates any interesting projects from this dataset, please do get in touch. Happy to answer any questions!