More or less. Estimating the depth from a single image is highly challenging and far from solved. We thus had to made sure that the depth estimate is suitable for synthesizing new views. And we are not explicitly modelling layers, we actually model the scene geometry as a point cloud. But you definitely got the gist of it. By the way, one advantage when estimating the scene geometry in a video is that it is possible to employ structure from motion.