We cannot be entirely sure of the behavior in any situation that is sufficiently different from anything that was presented during training. You can't necessarily expect a "common sense" response in that case, and in this sense the neural networks are black boxes.

The same applies to any complex system. 'Black box' means 'not inspectable', not 'anything which doesn't have predictable behaviour over all conceivable inputs'.