That is a very, very strong statement that requires some proof to go with such a strong statement of certainty.

While we can learn that the ball's change of movement just fine without swinging the bat, we can only do that because we are generalizing from a large body of knowledge that we devoloped by experimenting on the world using our body.

I am not aware of a single piece of evidence that an agent can use purely observational learning to ever aquire the causal knowledge of the real world to sufficient level to make those sorts of inferences with any sort of reasonable accuracy.