I feel like Multi-View reconstructions from a combination of high resolution ground and aerial photography are the only way we can hope to do better than Google Earth. Earth still suffers a lot on reversing lighting. Maybe a sunlight raytracing model could reconstruct neutral diffuse textures from a combination of time information, position information, and surrounding geometry.

Maybe some day we can get angular and spectral reflectance, normals, refraction, and subsurface diffusion maps. Each of these would be essentially the same optimization as the diffuse maps.