News for hackers. Stuff that matters.
Re:Automating programming: AI is transforming the coding of ... by YeGoblynQueenne on Saturday July 10, 2021 @12:30PM (#27794541)
interesting topic. Could you point me to some resources so I can learn more about it?

The following is a comprehensive introduction and survey of recent work in the field:

https://www.microsoft.com/en-us/research/wp-content/uploads/...

Regarding novelty, neural networks are excellent at interpolation but awful at extrapolation. What I mean by interploation is that the models learned by a neural network represent a dense region of cartesian space circumscribed by the neural net's training instances. The trained model can recognise and generate instances included in this dense region, including instances not included in its training set, by interpolation, but it is almost completely incapable of representing instances outside this dense region by extrapolation [1].

So basically what I'm saying above is that deep learning language model code generators can't extrapolate to programs that are not similar to the programs they've been trained on. Hence, no real novelty. But you can go a long way without such novelty hence my expectation that e.g. Copilot can be a nice boilerplate generator.

______________

[1] My standard reference for this is the following article by François Chollet, maintainer of Keras:

https://blog.keras.io/the-limitations-of-deep-learning.html

See also the second part of the series of articles where possible ways are proposed to address this limitation:

https://blog.keras.io/the-future-of-deep-learning.html

The latter is relevant to the conversation because the author basically proposes augmanting current deep learning approaches with a form of program synthesis.

There are no comments on this comment yet.