Tests can constrain a problem enough so a human programmer can understand what was intended. In theory, many programs will pass the tests, but in practice most matching programs will be convoluted and in poor taste - they only pass the tests and not the general rules implied by the test cases. In other words, there are additional constraints implied by common-sense principles of software engineering.

Developing an AI that can infer a simple, tasteful set of general rules (and suitable types) from a small set of test cases is certainly hard but not theoretically impossible.