Deep learning helps researchers remove clouds from satellite images

https://blogs.nvidia.com/blog/2021/09/23/gans-satellite-images/

by bcaulfield • 5 years ago

14 11 5 years ago

blogs.nvidia.com

TaylorAlexander• 5 years ago

I will never understand why people will write articles like this and not include any pictures.

mithusingh32• 5 years ago

Most likely because it doesn't work very well.

ma2rten• 5 years ago

The paper has some images: https://www.sciencedirect.com/science/article/pii/S147403462...

nreece• 5 years ago

There was a related article on 'Turning satellite imagery into wall art', that had pictures of easing cloud cover from photos: https://news.ycombinator.com/item?id=28325421

warkdarrior• 5 years ago

"we obtain reconstructed images that are more self-consistent"

The future is here. "self consistent" = "realistic" = "believable" = "convincing lies" = "fake"

asteroidbelt2• 5 years ago

Except satellite maps are not meant to be not-"realstics", not-"believable", not-"fake". They are meant to help with navigation, property purchase etc. If fake images help with that better than real images, that's fine with me.

toomuchtodo• 5 years ago

You cannot obtain ground truth from neural network guesses, even if those guesses are educated. What fills in the blank is speculation.

tornsandhog• 5 years ago

There is already technology that can see through clouds. Check out Radarsat run by the Canadian Space Agency.

sailingparrot• 5 years ago

Before everyones jump on the "GAN don't generate ground truth" bandwagon, that's not a concern here.

From what I understand, this is about creating a dataset of [aerial pictures, building mask] pairs, to then train a segmentation model for urban planning and design.

They use Unity and a virtual environment (a la flight simulator I guess) to generate a bunch of samples. They then take any image that has thin cloud covers, so clouds thin enough that you can still see the true outline of the buildings below it, and use the GAN to remove the clouds. Images with thick clouds just get discarded. So in this case the information that wasn't present in the original and that gets generated by the GAN does not matter for the task at hand (such as actual color and details of the building) as only the outline matters, and it was already visible before. That just helps the segmentation model learn by normalizing images in the training dataset.

What is not clear to me, is why didn't they just remove all clouds directly in the game engine since they are running in Unity instead of relying on a GAN.

aj7• 5 years ago

Are clothes next?

Deep learning helps researchers remove clouds from satellite images

Related Stories