Not a ditch, or a barbwire fence, or a telephone pole or a tree? I guess that may be tricky.

You can't reliably know the distance from pavement to any of those things unless you plan on indexing every physical object along every roadway.

I think the most likely scenario is self driving cars won't drive in those conditions, hopefully those conditions don't develop while you're on the roadways or you'll be parked somewhere cold.

They do index everything, the maps they rely on are lidar generated 3D point cloud fields that make a model of everything around the road, and are frequently updated.