Bing (via the Maps app) is also doing this type of thing, but with a less robust set of data.
Google's implementation appears to be based on oblique (angle, taken from plane) imagery similar to Bing's "Bird's Eye View" feature; if you open up old Google Earth you can see oblique shots from various angles that match the exaggerated coloring of the final 3D maps.
Seriously though, check out Apple Maps. I didn't think I'd be recommending it, as I nerd out about Google Earth a lot, but it's actually got great imagery & models.
The screenshots do look fantastic.
(Not a criticism of Apple if this isn't possible)
https://maps.apple.com/place?address=10365%20Gorenflo%20Rd%0...
There have been a couple small indie projects which basically load Apple maps in an iFrame, but I think they've been mostly C&Ded.
more info: https://www.youtube.com/watch?v=3apAXzf3JTg
You can wander around in this rendered copy of Earth, today, in VR. And folks are so above it that they still say "eh, no killer app tho" without irony.
Id guess the details of the process are tricky, but it's possible then you only need to concentrate on getting high quality images vs. getting high quality images and high quality elevation data at the same time.
I do wonder what exactly is new here in terms of a casual user browsing the landscape. Google maps 3D offers, more or less, the same experience. And that's been around for a while.
Still, it's hard to be negative about this. The result is quite clearly better than every other comparable technology I've seen.