Human perception of 3D geometry comes from motion. This is an algorithm to amplify motion. And that's really all that is needed. See "Structure From Motion" and "Optical Flow" for more information about this. Note that this is technology that is used in some video codecs. You may have noticed in some corrupted videos, the corruption artefacts can sometimes move as if they are mapped on a detailed 3d model of the moving objects. But really all it is are 2D motion vectors moving blocks of color around on a predetermined optical flow path.

In this algorithm, exaggerating that same 2D optical flow path has the perceptual effect of somehow "knowing" something about the 3d geometry. you could more accurately say the original motion is constrained by the 3D geometry.

(more) here is a kanye west video that uses the optical flow/video corruption effect intentionally for artistic purposes.

http://vimeo.com/4578366

see particularly the sequence starting at 1:00 to see unrelated background "texture" getting motion mapped onto kanye's moving face.

Thanks for the detailed explanation. The kanye video just looked like a corrupted low res video, I didn't understand that one. But i get the idea, its about extrapolation of moving pixels in 2D.

Same questions myself