Making Clouds Go Away on MapBox Satellite

http://mapbox.com/blog/improving-mapbox-satellite-by-making-clouds-disappear

by incanus77 • 13 years ago

140 27 13 years ago

mapbox.com

micro_cam2• 13 years ago

Cool. I hope they do it separately for winter and summer so as to give an idea of typical snow/ice cover.

celoyd6• 13 years ago

Author here.

We’re doing almost the opposite – as little non-permanent snow and ice as we can manage. The reason is simply that we want to make this as useful as possible as a general-purpose base layer, and people usually want to see spring/summer growth. It looks subjectively right even in regions where it’s only around for a minority of the year.

If you’re interested in seasonal dynamics, I recommend Blue Marble[0]. It’s really good work – we’re in touch with some of the people who made it, and they’re sharp folks. But it’s half the resolution that we’re aiming for, and has some areas of distracting interpolation artifacts. Our main goal is accuracy, but we’re also going for aesthetics in a way that Blue Marble wasn’t.

0. http://earthobservatory.nasa.gov/Features/BlueMarble/

PanMan• 13 years ago

In the article you say you took the pixels without clouds. Just looking at how white it is, or something more complex?

franck09• 13 years ago

looks amazing !

Will this be open-sourced one day ? Or maybe integrated into TileMill ?

crazygringo• 13 years ago

This was what I was curious about -- let's say, as you work your way up a mountain, the bottom never has snow, the middle has snow 50% of the time, and the top has snow 100% of the time.

How do you decide where the border is? You're going to have a snow cap of some size -- what size is it, and how do you get to look both realistic, and consistent-looking, without artificial-looking borders?

I do like your idea of it being summer everywhere, though, hemisphere be damned! :)

micro_cam2• 13 years ago

Thanks for response.

I think what you are doing makes sense for most purposes and I look forward to seeing the final layers.

I'm especially interested in snow cover because of a side project of mine (hillmap.com) that is targeted at backcountry travelers (ski tourers, hikers etc...especially winter travelers since it does things like avalanche terrain analysis in a canvas overlay).

Blue marble is cool and there are some overlays from NOAA etc but winter satellite photos good enough to answer questions like "is this typically an open snow field or heinous bushwhack" or "does this lake freeze over" would be really useful for planing trips in winter or colder climates.

If you guys (or someone) hosted the daily datasets it would be possible to do the processing client side in a canvas for a user defined time periods which would be an awesome tool.

Houshalter2• 13 years ago

Wouldn't the clouds still interfere with the averages though? That is areas that have lots of clouds would seem whiter than they really are? It seems to look alright though.

dsl• 13 years ago

They detect the images with clouds, then average the rest.

Houshalter2• 13 years ago

Ok that makes sense. I thought they were using this as a replacement for filtering out pictures with clouds.

brunosan• 13 years ago

Wouldn´t the sun-synchronous orbit allow you to use a similar a pipeline with local-time filtering to create a night-time map? That would be quite amazing, and also useful for socio-economic folks (or even correlate that with OSM coverage)

I could only find a related talk about this here: http://modis.gsfc.nasa.gov/sci_team/meetings/199905/presenta...

ghshephard• 13 years ago

The maps look absolutely beautiful. Can I suggest you drop Apple a call and offer your assistance? I just came back from Singapore, and good portions of their hybrid maps were basically unusable because of cloud obstruction. (To be fair, Google also has a bit of cloud coverage of Singapore - but their hybrid maps were completely usable).

Cloud obstruction of satellite imagery six months after Apple released their mapping data, seems to indicate a fundamental lack of knowledge of how to deal with those type of obstructions, so hopefully your call will be well received.

huhtenberg• 13 years ago

Got a number?

Demiurge2• 13 years ago

This is interesting, but very low on detail. For example, was another modis product used for clouds or was it based on a simple EVI/water cloud mask? Is this MOD09A1, 500m or 250m? Lastly, why or how is this MODIS layer going to compliment existing Landsat or DigitalGlobe?

jcr• 13 years ago

The images are amazing. When looking for clear skies, I was curious if you saved any metrics regarding how often a particular location (pixel) is clear or cloudy?

(NOTE: I have no intention of visiting the cloudiest place on earth with my vampire hunting kit. ;-)

You might find the weather simulation work done on the old (2005) "Earth Simulator" or newer (2011) "K Computer" supercomputers in Japan really interesting.

http://www.hpcwire.com/hpcwire/2011-06-20/japanese_supercomp...

celoyd6• 13 years ago

I was curious if you saved any metrics regarding how often a particular location (pixel) is clear or cloudy?

We didn’t, but it would be pretty easy to re-derive. The cloudiest places at this scale are the sides of mountains in the Intertropical Convergence Zone[0], where wet air rises several thousand meters, thus cooling and thus condensing. For example, tomorrow I’ll be spending a lot of time looking at Andes in Colombia, Ecuador, and Peru.

0. http://www.youtube.com/watch?feature=player_embedded&v=Y...

noiv• 13 years ago

shamelessplug: Actually MODIS scans Earth using different wave lengths. The near infrared channel can be used to detect clouds and even to distinguish them from ice. Using a little recursive AviSynth Script I was able to produce an animation of daily changes in the European part of the Arctic with last year's data. It starts with all the clouds and then adds each day's cloud free pixels. The goal was to visualize how Kara Sea melted out within a few days. Enjoy: http://www.youtube.com/watch?v=j-EDFs8f_78 /shamelessplug

vladtaltos• 13 years ago

looks pretty cool. great work. I especially loved the fact that your mehod does not blur the edges significantly and the color profiles are pretty good.

I have a few questions you haven't mentioned in your blog and would appreciate it if you can answer them here.

is there some sort of public ftp server you can log in and get say the last 2 days' maps of any given coordinate ? and if so can you share the address ?

additionally, are the image coordinates that come with the images really that accurate that you can do pixel comparisons over time without doing any sort of registration ?

jre• 13 years ago

This is pretty cool !

I worked on a similar problem during my bachelor thesis[1] : I wrote an algorithm to remove clouds from NDVI images (used for deforestation detection[2]). We used fourrier, moving windows and some ugly hacks to detect and interpolate cloudy data. There are some details in my report.

[1] http://ape.iict.ch/teaching/DiplomaReports/2009_Rebetez.pdf

[2] http://terra-i.org/terra-i.html

Making Clouds Go Away on MapBox Satellite

Related Stories