If I understand correctly this means we should just write small program that can learn which reactions to given stimuli of the program are good, and which aren't, based on history of signals and responses of the program that are known to be good.

While interesting, I doubt it will do much to make programs more correct.

Somebody has to tell the program that this new reaction is good, so you should learn it as an exception to the older rules, or it is wrong, and we have found a bug.