They could stream a spherical texture (with depth map). Locally you would only have to render the sphere from the view point of the eyes.