Needing more training data seems wrong. Imagine the space of all programs. Can we agree that one can only train a model on a given subset of that space? Now, imagine wanting a program P that solves a particular problem, and that the program P is not in the training data. How does more training data guarantee that the output is P and not a neighboring P' that is almost correct, but subtly different and therefore utterly wrong? I think (and the author seems to agree) that the most we can hope for is a less sloppy approximation, not a correct program.