> Flat patches of color have no features and as such, cannot be correlated between cameras.

Flat patches of color are also flat, which makes it possible to fill in the missing depth information.

I took a course in computer vision where one of the projects [1] involved monocular vision, and assuming that flat patches are flat, vertical lines are vertical, all others are horizontal and the background is flat, it was possible to get a pretty good reconstruction.

[1] http://groups.csail.mit.edu/vision/courses/6.869/notes/chapt...