Most of early physics was done by curve fitting, and on very small datasets. Simple Newtonian dynamics shouldn't be much of a hurdle for a neural network. That's about as much anticipation of cause and effect as you'd need for faking most kinds of videos.