Re:Deep learning helps researchers remove clouds from satell...
by Voloskaya on Thursday September 23, 2021 @09:06PM
(#28637516)
Before everyones jump on the "GAN don't generate ground truth" bandwagon, that's not a concern here.
From what I understand, this is about creating a dataset of [aerial pictures, building mask] pairs, to then train a segmentation model for urban planning and design.
They use Unity and a virtual environment (a la flight simulator I guess) to generate a bunch of samples. They then take any image that has thin cloud covers, so clouds thin enough that you can still see the true outline of the buildings below it, and use the GAN to remove the clouds. Images with thick clouds just get discarded. So in this case the information that wasn't present in the original and that gets generated by the GAN does not matter for the task at hand (such as actual color and details of the building) as only the outline matters, and it was already visible before. That just helps the segmentation model learn by normalizing images in the training dataset.
What is not clear to me, is why didn't they just remove all clouds directly in the game engine since they are running in Unity instead of relying on a GAN.
Re:Deep learning helps researchers remove clouds from satell...
by asteroidbelt on Thursday September 23, 2021 @08:27PM
(#28637269)
Except satellite maps are not meant to be not-"realstics", not-"believable", not-"fake". They are meant to help with navigation, property purchase etc. If fake images help with that better than real images, that's fine with me.
Re:Deep learning helps researchers remove clouds from satell...
by asteroidbelt on Thursday September 23, 2021 @08:55PM
(#28637442)
If you are planning to catch Osama bin Laden, then you probably can't rely on speculations.
But if you try to plan a good hiking route, guesses might help you spot something interesting which you wouldn't notice otherwise because the image is noisy.
From what I understand, this is about creating a dataset of [aerial pictures, building mask] pairs, to then train a segmentation model for urban planning and design.
They use Unity and a virtual environment (a la flight simulator I guess) to generate a bunch of samples. They then take any image that has thin cloud covers, so clouds thin enough that you can still see the true outline of the buildings below it, and use the GAN to remove the clouds. Images with thick clouds just get discarded. So in this case the information that wasn't present in the original and that gets generated by the GAN does not matter for the task at hand (such as actual color and details of the building) as only the outline matters, and it was already visible before. That just helps the segmentation model learn by normalizing images in the training dataset.
What is not clear to me, is why didn't they just remove all clouds directly in the game engine since they are running in Unity instead of relying on a GAN.
The future is here. "self consistent" = "realistic" = "believable" = "convincing lies" = "fake"
But if you try to plan a good hiking route, guesses might help you spot something interesting which you wouldn't notice otherwise because the image is noisy.