Creating Heatmap from Scratch in Python (2018)

http://www.geodose.com/2018/01/creating-heatmap-in-python-from-scratch.html

by geomatics99 • 7 years ago

70 23 7 years ago

geodose.com

gigatexal• 7 years ago

This is really cool. Thank you for sharing.

geomatics992• 7 years ago

you're welcome

yetihehe2• 7 years ago

I much prefer kriging, which seems to better model sparse areas. https://en.wikipedia.org/wiki/Kriging too bad there are no image comparisons for those methods. Main difference - instead of sum of distances with cutoff, you make sum of values weighted by distances for each map point.

plopz• 7 years ago

What you describe sounds more like idw than kriging.

yetihehe2• 7 years ago

Indeed, I've started with nice images on web which were described as "kriging", then went on to page about kriging, seen some general ideas and implemented what I thought would be fast, simple and work. That way I had something which looked right for my application (visualising district heating temperatures in city), but only now I know how it's really called. Still better looking that heatmap.

4thaccount2• 7 years ago

Neat...I just wish there was an easy way to give the basemap library a 3-item tuple with (lat, lon, value) and have it automatically do a heat map and not just plot the points. That is an option in some software.

urschrei• 7 years ago

basemap (like matplotlib) has the contourf method. You can calculate your x, y, and z values using matplotlib.mlab.griddata (altho scipy's are faster), and pass the results to it:

https://stackoverflow.com/a/26885815/416626

geomatics992• 7 years ago

Thanks for the link

anc843• 7 years ago

Related: basemap is deprecated, use cartopy instead! https://scitools.org.uk/cartopy/docs/latest/

kaybe2• 7 years ago

cartopy's website assumes that I know what I am doing.

There is no good entrypoint into the documentation, it just starts to tell me about projection details. The documentation outline is a list of the code functions with no clear point to start or indication which places might be good to start at.

I also can't find any examples for quick demonstration and trying. (edit: I found them after clicking all the links. Well... it's in the middle of a list of details I'm not interesting in and says 'Using cartopy with matplotlib'. Oh, at the bottom there is a link with more examples which I could find neither in the index nor in the outline... urgh.)

basemap's website also has a lot of problems, but it is a lot easier to use and we are lazy.

rakshitadmar• 7 years ago

I have something of the sort. Give it later, lon and a value, it performs cubic interpolation and forms a heatmap. Some of the code is specific to my application but I'm sure you can reuse.

https://github.com/rraks/sigcatch

Open to contributions.

4thaccount2• 7 years ago

Thank you! I'll look.

anc843• 7 years ago

Please be aware that you MUST NOT use this for geographic coordinates such as "GPS" or "WGS84" latitude & longitude pairs. Depending on the longitude areas are of different sizes. You must use code that knows about that or your results will be utterly wrong.

peteradio• 7 years ago

This would only skew density maps not counts maps.

anc843• 7 years ago

Well, this is about density maps unless I overlooked something?

kuzehanka• 7 years ago

How does this factor for tools like Datashader?

aeroevan2• 7 years ago

Same idea for datashader.

If you're overlaying datashader on a map, use the projected coordinates as input (to both datashader and/or any heatmap).

mturmon• 7 years ago

The OP makes a related mistake (although for their data, it worked out OK). It uses this grid:

  x_grid=np.arange(x_min-h,x_max+h,grid_size)
  y_grid=np.arange(y_min-h,y_max+h,grid_size)

where h is a small constant, and the range ends are the min/max of the data. In general, this will result in rectangular pixels, which you probably don't want.

In your example, even for a small geographic area, a degree of longitude will typically translate to a smaller distance than a degree of latitude, so as you say you can't treat them the same.

alexhutcheson2• 7 years ago

I like seaborn's jointplot()[1] function for generating plots like this. I actually prefer the "hexagonal bin" plot, but jointplot() can also do kernel density estimation like this post does.

Hexbin example: https://seaborn.pydata.org/examples/hexbin_marginals.html

KDE example: https://seaborn.pydata.org/examples/joint_kde.html

If you are doing any exploratory analysis using scatterplots, I highly recommend that you also generate hexbin plot or heatmap. Far too often it turns out all of the points are concentrated in a small area of the plot, but that's not always apparent on a scatterplot.

[1] https://seaborn.pydata.org/generated/seaborn.jointplot.html

alexhutcheson2• 7 years ago

If you're making heatmaps with GIS data, give some critical thought to the interpretation of the map to make sure you're not just doing this: https://xkcd.com/1138/

KyleOS• 7 years ago

I feel like these days we should be creating more user-friendly graphs - both for the creator of the graph and for the reader. I created this graph of the Atmospheric concentrations of CO2 earlier today (https://kyso.io/KyleOS/atmospheric-co2-concentrations#code=h...). Would love to see the OP's heatmap recreated with plotly.

Creating Heatmap from Scratch in Python (2018)

Related Stories