Yup, in ML you would call this the generalization problem: How are you going to build your algorithm build something useful instead of just memorizing the inputs in some way?

Differentiable Neural Computers [1] are getting us closer (by incorporating an explicit I/O operations on a memory) but it's uncertain how much test data you would need to build complex code routines.

https://deepmind.com/blog/differentiable-neural-computers/