If the input is numbers then what happens is simply extrapolation (or regression etc). If the input is symbols, then what? The fact that you can associate numbers to the symbols does not help. The extrapolation will be meaningless. Unless you happen to have the correspondence '1'->1, '2'->2, etc,

How are you defining symbols? How are you defining meaningless, or meaning at all for that matter?

Suppose you have a function f of two variables and all you know about it is that f(x,y)=x+y for a few values of x,y. With this data, the computer extrapolates linearly and finds f(x,y) for _all_ values of x,y. The computer just "learnt" how to add! If all you have is symbols, however, you have to have numbers associated with those symbols. But you can't expect 1 correspond to '1' and 2 to '2'. As a result, your function f - and its extrapolation - will have nothing to do with addition (f won't even be linear).

So am I correct in assuming that you are looking for a machine that can do the mapping of a symbol into a form that the function can learn from? Your visual and auditory systems do this, and since they are physical systems this mapping is computable.

By the way, for the addition example , the neural network needs only one example, e.g. 1+1=2, applied repeatedly to discover the addition of any two inputs. There is no extrapolation between examples needed in this case.

"Extrapolation is the process of constructing new data points outside a discrete set of known data points."

I have rethought what you have said, and I think I am now closer to understanding what your question is. You are posing the question of what to do with an input, not the operation itself. For instance, if I feed the network an image of 5 and 3, with the output being the image 8, the network would not perform addition but some other functional mapping. Is that the feature you are suggesting that it should 'understand'?

Yes.

Right. Confusing extrapolation with interpolation.