It doesn't have to be just NN's. For example, stereoscopic vision (triangulation) or LIDAR can create a 3D depth map model (perhaps with certainty weights). NN's can also create one based on mass 3D training on flat images (non-stereo). The two models can be compared, and alternative pattern-match candidates can be explored for the spots that don't match well.

I suppose the stereoscopic vision system could use NN's, but the implementation can be swapped for something else as long as the result (interface) is a depth map. You could switch it from LIDAR to multi-eye stereoscopic translation and vice versa. The interface between the parts is something human analysts can inspect and relate to.

You may also have a 3rd module that attempts to model the scene using 3D rendering of the "guessed" objects (using a database of known objects) from the flat-image NN and depth-map first pass, using the match/non-match feedback to test yet more guesses. (Certain objects with semi-random shapes like a pile of cloth or mountains may be excluded from known-object-matching, at least as a test candidate.)

There might even be a genetic algorithm to "breed" the best 3D-rendered model that fits the depth map, the flat image, and the top flat-NN candidates.