I wonder if we’ll eventually see a system where instead of writing code you describe in natural language what you want the program to do and then ML is applied in order to generate the code for that.
I mean, a lot of people have been interested in the past in making human programming languages, and had varying degrees of success.
Personally I love writing code but, it could be, couldn’t it?
Write some unit tests, a human description of what it does and based on the source code and description of existing software the system would basically “debug the program into existence” for you.
That’d be kind of freaky, kind of cool and a little bit scary.
The problem with programming is not the encoding of the requirements in programming language for the most part. The problem is that the specifier (A in this example) usually does not have a full grasp of what they actually want. In fact, they usually don't have any idea at all. "Give me a e-shopping system to sell comic books" is the level of detail they can understand.
The closer A can come to expressing the requirements they need, the closer they are to actually being B in reality. B's real skill is not in knowing the syntax and grammar of the computer language, it's in knowing that in order to make a system that will satisfy A we need to do X, Y, and Z to the tiniest detail.
When we get into trouble with our software is when we write code that is dramatically more complex than the problem we are trying to represent. This doesn't happen so much because we don't know how to program. This happens because we are slowly extending the code base over time with imperfect knowledge about what we are ultimately building at any given time. We also have to trade-off the benefit for getting something done with discovering generalities that allow us to simplify the expression of code that we already have.
I don't think we will ever replace "programmers" with AI -- at least not until the AI can be trained to ask the important questions about what the system really needs to be (and for that we need turing-test passing level AI). I think it's much more likely that we will build more and better tools that help programmers visualise and plan the programming situation. I think we will have automatic code generation because we already have it: Look at "derive" in Haskell and Rust, for example. But I think that's the level of automatic code generation we're going to want for at leas the next 20 years or so.
Interestingly for testing, I think we'll actually go the opposite direction: We will spend more time thinking about requirements and the computer will help us by writing tests that challenge our assumptions: I've broken this function, are you sure you got it right? Again, we already have these kinds of systems and I think that this is the most appropriate direction to invest in research.
This is already possible, but not with deep learning which is probably the reason you haven't heard of it.
Learning programs from specifications (not necessarily in natural language) is the subject of the field of Program Synthesis [1].
Learning programs from examples of their input and outputs (which is basicaly writing unit tests) is the subject of Inductive Programming [2].
Inductive Programming encompasses the fields of Inductive Logic Programming and Inductive Functional Programming, that target logic and functional programming languages, respectively.
You won't find anything learning from examples in javascript or python directly, though- imperative languages are too sloppy for that sort of thing.
___________
Sounds more or less like the mechanism by which developer jobs succumb to automation. Hopefully those of us that are working class have seized the capital by then.
To spin that more positively, it may automate the basic boring stuff (both for us technical types and potentially for the simian on the street) and leave us more time to spend time in more fun & challenging playgrounds.
Microsoft was showing off similar work from their own code datasets this year at ICLR, I couldn't find a link online, but the demos had block suggestions from method signatures for C#. It should be possible to get similar results with natural language queries.
(thanks Nick. Here are the links)
Generative Code Modeling with Graphs: https://arxiv.org/abs/1805.08490 Learning to Represent Programs with Graphs: https://arxiv.org/abs/1711.00740
I do think there would be value in machine-written code if it made code in general more alike, so that you don't have to relearn the weird tricks any new writer could have chosen to use.
There wasn't a technical reason (unless you count lazyness as a technical reason) -- we simply had infrastructure for Python specifically lying around from past research projects, which we initially reused.
After we got Nat's feedback, we redid our data processing pipeline completely to be based TreeSitter (which wasn't around when we started thinking about parsing Python), which makes it much easier to scale to the number of programming languages on GitHub.
We've seen huge advances in human language modeling and translation due to the success of deep learning. Often new directions start with a really motivated team producing a new kind of dataset. Who better to do that for code as language than Github!
This started as a grassroots effort inside of Github, and went through many iterations. When it was presented to Github's CEO six months ago, he correctly pointed out that we needed to go back and include Github's most popular language (javascript). As the project went on many smart people chipped in, and we produced something that we think is truly useful.
Check out the paper here: https://arxiv.org/abs/1909.09436
We overcame plenty of challenges to pull this off. For example: how do you clean this data? how do you label it? We've got folks from Github, Microsoft Research and Weights & Biases here to answer any and all questions you might have. Can't wait to see where this goes!