You use a bunch of cameras, calculate a depthmap, and use the spherical video and the depthmap to apply parallax, like this [0, 1] but spherical. It's a kind of faked stereo video.

Google Jump is one such camera jig that use 16 cameras [2].

[0] http://depthy.me/ [1] http://www.scalari.net/2008/04/22/generating-stereoscopic-im... [2] https://www.google.com/get/cardboard/jump/

You record from two sets of cameras positioned an average eye distance apart, then project one set of cameras to the inside of a sphere shown to one eye, and the other set to the other eye.

The limitation is that the cameras are arranged horizontally (current gen), so the stereo effect is lost if you tilt your head, as well as at the north/south poles of the spheres.