Show HN: I shot high-res stitched panoramas in Iceland using a thermal camera

https://petapixel.com/2019/07/13/shooting-high-res-thermal-photos-of-iceland-to-show-nature-at-work/?q=5

by dheera • 6 years ago

753 95 6 years ago

petapixel.com

supernova87a2• 6 years ago

If only the cost of thermal imagers could come down and be more affordable, more people might be interested!

jcims6• 6 years ago

That and the associated optics.

There’s some reason for encouragement however. I bought a Seek Pro, with 320x240 resolution, on sale for $350 earlier this year. The app is pretty terrible unfortunately and the camera physical construction is flimsy, but the imager seems to be of reasonable quality for the price point.

Couple of examples: https://imgur.com/a/ZGsOwUm

ampdepolymerase• 6 years ago

Thermal imaging resolution is restricted by arm control laws unfortunately. There are forces at work beyond market forces.

jansan• 6 years ago

I once rented an IR camera for roughly 100€. This is probably the best option for people who only need IR cameras on very few occasions.

Will we ever see chap(ish) IR cameras? Is there an alternative to germanium lenses on the horizon?

giantg26• 6 years ago

That carboy is pretty hot.

garaetjjte2• 6 years ago

Seek cameras are rather noisy though. More expensive, but much less noisy and still reasonable to buy as a expensive toy would be TE-Q1 from i3system: http://nicesuper3.cafe24.com/down/TE-Q1-compressed.pdf

gandalfian2• 6 years ago

Yes I do covet them. Getting cheaper but still fairly low resolution. Keep hoping for a tipping point like digital cameras where cheap decent sensors will suddenly be everywhere. Not yet though.

jstanley2• 6 years ago

I bought one on eBay a few years ago for about £300, but it sucked so I got rid of it. It used a traditional camera to draw most of the image (itself already quite low-res), and just highlighted even-more-low-res temperature data in blue and red.

SoSoRoCoCo3• 6 years ago

I was just considering buying one. Would you share your thoughts?

soared• 6 years ago

I'd imagine demand for thermal imaging has skyrocketed due to covid, so its possible the tech will get better and cheaper.

CyberDildonics• 6 years ago

I don't think cheap cameras are sensitive enough and you don't need an entire image that is better for seeing relative heat, you just need a single accurate reading. Those thermometers are only $20 and are being cranked out in bulk.

SoSoRoCoCo3• 6 years ago

It's $570:

https://www.amazon.com/Seek-RQ-AAA-Thermal-Revealpro/dp/B07B...

gandalfian2• 6 years ago

Cunning. He took panoramas with thermal camera and simultaneous black and white photos and coloured the b&w photos with the data from the thermal camera. Interesting examples of insulation. I wonder how the result compares to the standard Flir auto blended images. Msx?

brk• 6 years ago

This would be better than FLIR's MSX, which generally suffers from a lack of precise alignment of the thermal and optical images, particularly for shots with a lot of depth of field.

giantg26• 6 years ago

Sounds like an opportunity to sell the algo to flir

osamagirl694• 6 years ago

I have done a lot of work with high resolution thermal imagers and never seen any images that look like these so I was curious what was going on.

I also simultaneously shot a black-and-white photo with a regular camera and a wide-angle lens for comparison. I then wrote my own program to color the black-and-white photograph with the actual thermal data, using a false-color scale which you see in the upper-right corner of every image.

Amazing drawings, but these are not actually thermal images.

mmaunder5• 6 years ago

Pure thermal would have been quite smudged. IMHO the B&W adds really nice context - and it sounds like it's not just a straight overlay, but he's recoloring the B&W pixels, which I think provides a crisper result.

I'm curious what resolutions you've worked with, if you don't mind sharing. High frame rate thermal cameras (above 9hz) are ITAR restricted. Hi-res above 640x480 is not common on the consumer market and is restricted to embargoed countries. Cooled thermal cameras are restricted too IIRC.

osamagirl694• 6 years ago

The device we had was 1.3MP--rough the same resolution as the composite image discussed in this article. The camera core was expensive, but the lens actually cost more than the sensor. Yes it was ITAR regulated.

As an example of what I would have expected, this product brief (not the device we were using, but similar) has a picture of the white house https://www.lockheedmartin.com/content/dam/lockheed-martin/m...

_0w8t2• 6 years ago

At my previous work I was involved with processing of data from 1024x800 cameras with 16 bit per pixel and nice optics. After proper calibration and trivial image processing they were not particularly blurry since they were able to resolve even tiny temperature differences.

slavik81• 6 years ago

ITAR restricts the export of arms technology from the United States, but the big names in traditional digital cameras are all Japanese. Perhaps naively, I would have thought they'd have the expertise needed to build a high-quality thermal camera without US involvement. I suppose infrared imaging must be quite a bit different from visible light.

andreareina• 6 years ago

Don't commercial thermal imaging cameras also do a thermal/visible composite? Would those also not count as thermal images?

dheera8• 6 years ago

OP here -- I added a top-level comment with some more links. In particular you can find the separate thermal and visible images on the repo: https://github.com/dheera/iceland-thermal

osamagirl694• 6 years ago

Thanks for the clarification! This makes sense, the reason your images look 'off' is that the normal thermal pallets (such as the one in your colorbar) are typically a roughly constant (or at least smoothly varying) luminance with the thermal information stored in the chroma. Your images have that multiplied by the visible light image intensity which gives a very surreal effect. Fun stuff

caymanjim• 6 years ago

By that metric, most of the NASA images you see are "drawings" as well. I worked on imaging systems for various NASA spacecraft, and most of the published images are false-color, albedo-adjusted, averaged mosaics of many imaging sources from different orbital passes over months or years. Images of people and everyday things are heavily-processed as well, using techniques like HDR and color correction. Photography has always involved this sort of technical manipulation.

Yajirobe• 6 years ago

By that metric, most of the NASA images you see are "drawings" as well.

Yes, there is no debate about that.

Razengan• 6 years ago

Even biological vision involves some processing from raw light → mental perception.

mmaunder5• 6 years ago

I'm curious what kind of thermal camera he's using.

I bought a Therm-App Pro 640x480 a while back and it's quite good - and reasonably priced compared to Flir - along with a github repo for different false color palettes. But what's really weird is it seems to not be available anymore. I'm guessing export restrictions or massive demand for thermal cameras during Covid along with supply chain issues. IIRC it is also 25 frames a second, which is unusually high for thermal cameras due to export restrictions.

Edit: Here's a post on creating panoramic images with therm-app. https://www.flickr.com/groups/therm-app-users/discuss/721576...

bobbob1921• 6 years ago

I was wondering this too, the author added this info via a reply in the comments (of his site/article , not HN)-

Author here. The camera was a Seek RevealPro which has 320x240 resolution. The thermal data in the panoramas are stitched together from raw data exported by the camera and represented as color (hue+saturation, as per scale in upper right, represents temperature data).

The brightness channel is piecewise-linearly combined with a corresponding visual image taken with a regular SLR to provide a visual reference of what you're looking at.

mmaunder5• 6 years ago

I think the ThermApp cameras had the ability to combine a visual photo, but I haven't used mine for quite some time so I may be wrong.

dheera8• 6 years ago

OP here. Seek RevealPro. I added a top-level comment with some more details and links :)

mmaunder5• 6 years ago

Thanks OP. Really great work on those Iceland pics. It’s an original concept I haven’t seen before - adding thermal data to panoramas of landscapes with geothermal heat. So cool. I’m a bit jealous I didn’t think of it first.

Sounds like you have a science background and field, but I’d encourage you to think of these as art and chat to a few galleries about an exhibition. At the very least it would be fun, and pretty amazing to see big blowups of your own imagery. They may be able to do a virtual exhibition during Covid.

jacquesm3• 6 years ago

Thank you for all of these, awesome work.

giantg26• 6 years ago

Looks really nice. I see one for sale in the US. $4k though.

joshvm2• 6 years ago

You can buy a FLIR Boson at 640x512 for that price. The cost of thermal cameras is roughly linear with the number of pixels. I realise that's not cheap, but you can get double the resolution (e.g. 4x pixels) for the same money.

centimeter2• 6 years ago

I wonder how much imaging technology has been held back by ITAR.

ahmadhamza19• 6 years ago

Really amazing photos.

cs702• 6 years ago

Beautiful. Eerie. Great work. Thank you for sharing.

Your images look almost like they came out of James Cameron's vivid imagination. If you had told me these were computer-generated images of Pandora at night[a] from the upcoming Avatar sequel, I would have believed it.

[a] E.g., see https://www.google.com/search?tbm=isch&q=avatar%20pandora%20...

dheera8• 6 years ago

Hi everyone, OP here.

I used a Seek RevealPro which outputs TIFF files that contain 32-bit floating point values. I created a script that takes in a Seek TIFF and outputs an 8-bit greyscale integer TIFF:

https://github.com/dheera/iceland-thermal/blob/master/script...

I then panoramically stitched those 8-bit TIFFs together (some with the help of hugin, and some manually with GIMP) and distortion-corrected them as best as I could to a black-and-white visible light image. I also created an artistically-chosen color map for each image. For the final output image, hue and saturation are fully based on thermal data, and lightness is a combination of thermal and visible data. Scripts to generate the final output images, along with the panoramically pre-stitched data, and the color maps I used:

https://github.com/dheera/iceland-thermal

Also, my Instagram, which is mostly astrophotography of late, but I did post thermal images there before and I may do so in the future again if I can find another good set of interesting thermal subjects to do this in!

https://instagram.com/dheeranet/

skybrian• 6 years ago

Have you tried any other thermal cameras and how did they compare?

dheera8• 6 years ago

Not too many, for personal projects I'm mostly budget-constrained. I tried the Flir C2 but the thermal data was extremely low resolution, and returned it. I also tried a Flir Lepton 2 module since I could use an RPi and some gadgetry to do the panorama scanning and stitching automatically, but it too was low resolution (80x60).

The Lepton 3 seems to be decent at 160x120 though, which is probably good enough for a project like this, so if I were doing this again I'd probably use a Lepton 3 and a 2-axis servo scanner with a RPi, and maybe even combine it with an RPi HQ camera to automate the entire process into a single push button.

The Seek RevealPro is slightly annoying in that it doesn't have programmatic access so I had to press a button to take every piece of the panorama. It's a great general-use device though. I use it when camping or doing astrophotography in remote locations for some peace of mind since I can spot any potential wildlife and people threats from a distance, and I can easily identify whether a parked or seemingly abandoned car has been recently occupied. (My fears may be unfounded, but either way I'm much more at ease when I can see in the dark.)

tpmx4• 6 years ago

That photo with the icelandic horses - how did you pull that off with a QVGA-resolution thermal camera + stitching? Animals tend to move...

dheera8• 6 years ago

Yeah so I tried to get that shot about a dozen times on various farms before I finally found some horses that seemed tired, lethargic, and didn't move too fast.

I also had to stand there for a good 15 minutes in the biting cold before they stopped being interested in me and kind of just stood there.

They still occasionally turned their heads though so I quickly snapped my visible light image with a DSLR in one hand on a tripod and scanned over the horse part with the thermal camera in the other, before patiently scanning the rest of the scene. The horses in the distance though still did move, as you can see in the separate thermal/visible images in the repo.

I had to do a similar thing for the erupting geyser. 2 quick thermal shots in rapid succession for the eruption in one hand, DSLR in the other, then scanned the rest of the scene. Took about maybe 20 tries at least.

deadw3ight• 6 years ago

That's sick! Brilliant and creative idea. Those shots look beautiful.

ChrisMarshallNY• 6 years ago

Awesome stuff! Thanks for sharing it!

I'm used to thermal cameras being so lo-res you can barely tell what they are.

brianjunyinchan• 6 years ago

Love the beautifully haunting perspective.

systemvoltage• 6 years ago

Why are Thermal cameras lagging in resolution? Visible wavelength cameras can be had upwards of 50MP for $5k.

0xFluegel• 6 years ago

The sensor needs a lot more energy. This produces two types of IR cams: actively and passively cooled. The passive cooling is easy to take around with you but only possible with a smaller sensor.

As for the price per pixel: I don't know from memory.

semi-extrinsic2• 6 years ago

I think it's a combination of physics (wavelength is 20x larger than visual light, making optics and sensors trickier) and export control, as you could (literally) weaponize a high-res thermal camera to make an autonomous drone/missile targetting hot exhaust etc.

jzwinck2• 6 years ago

Thermal cameras measure much larger waves (7-14 microns as described in the article), and their sensitivity to amplitude is high.

A major limiting factor in camera sensor resolution is signal to noise ratio. It is well known that if you bump up the "ISO" (light sensitivity) to make photographs at night, you see more noise. The higher the sensor resolution, the smaller each photosite and therefore the less signal (fewer photons) captured in the short time of one exposure.

Larger sensors can solve this problem but these are CMOS devices whose cost increases superlinearly with area (like microprocessors, see die yield). Most customers won't accept dead pixels.

Economics is also a major enabler of kilobuck 50 MP visible light cameras, miracles though they are. Industrial users aren't going to re-buy all their cameras every few years like DSLR users do.

jedimastert• 6 years ago

I wonder if lower-iso long-exposure thermals would be useful, or if it would get too "motion blurred"

supernova87a2• 6 years ago

As I recall, IR sensors are also much more expensive to produce, as in, 1Kx1K is already a very large array and costs thousands of $. So any ordinary consumer is not likely to be using something with very high resolution.

giantg26• 6 years ago

I agree. The median income in the US is $35k, and many of these things are 5-10% of that.

I'd really like a monocular, but anything decent is very expensive.

tpmx4• 6 years ago

This is just slighly informed, please prove me wrong.

The major market so far seems to have been tactical/military, so they have focused on framerates rather than resolution. And then the US blocked high framerates for civilian use...

(edit: the US blocked civilian exports.)

RyJones2• 6 years ago

My thermal camera[0] is 30Hz, which I assume is 30 FPS, and I'm a civilian.

[0] https://www.bhphotovideo.com/c/product/1194041-REG/flir_431_...

chris_overseas• 6 years ago

I bought a HIK Vision Vulkan 35mm Pro [0] in the UK recently which refreshes at 50Hz, and I've seen others that can do 60Hz. One nice thing about the HIK Vision is that it comes with a documented API for controlling it via a wifi hotspot.

[0] https://www.hikvision.com/en/products/Thermal-Products/Comme...

rahimiali• 6 years ago

Visible wavelength cameras rely on relatively cheap CMOS sensors. CMOS processes have been squeezed for efficiency for decades to make, for example cheap DRAM.

The technologies behind thermal cameras are very different because they need to be sensitive to wavelengths 10x longer than visible light (the silicon in CMOS is basically completely insensitive to light above 2 um, whereas thermal cameras need to image at ~10 um). The pixels are effectively microbolometers and rely on more exotic materials like Germanium.

jacquesm3• 6 years ago

Funny how once upon a time it was Silicon that was considered to be the rare material for semiconductor manufacture. Then it took over and Germanium was pretty much lost, though at 0.2V forward bias it has a lot of unique applications in sensing gear. It is also why you really want to use a Germanium diode for your crystal radio set!

_trampeltier• 6 years ago

Because companys pay so much, so why make an cheaper product.

Mike did a real teardown on a Flir E4 many years ago, and saw (with an oscilloscope), even on the cheap camera with the low resolution, the chip has the full resolution. I think one week later the camera was hacked.

https://www.youtube.com/watch?v=NtqUE67BUDI

centimeter2• 6 years ago

Microbolometer pixels are big and expensive compared to CMOS pixels.

pachico• 6 years ago

From what I know, it's hard not do do awesome pictures in Iceland. What a marvelous place...

NDimCube• 6 years ago

Nice results, thermal cameras aren't easy to work with, especially in terms of artistic result. I've seen problems with stitching resulting with artifacts in panoramas. As I understood (I'm not thermal cameras expert) it's due too the fact that sensor warms up -> thermal camera calibration doesn't correct the result enough -> stitching needs corrections.

To put it one dimension higher, in Pix4D we work on 3d models based on the thermal images: https://www.youtube.com/watch?v=x-jp9a1bpVU Example: https://sketchfab.com/3d-models/thermal-model-of-pix4d-headq...

kbutler2• 6 years ago

These are beautiful.

Any thoughts on why the photographer reversed the spectrum (red cold, blue hot) for the fumarole? https://petapixel.com/assets/uploads/2019/07/image10-1.jpg (just looked cooler?)

Or why the distant horses have breath showing, but the closer ones do not? https://petapixel.com/assets/uploads/2019/07/image1.jpg (maybe a resolution/compositing issue?)

dheera8• 6 years ago

Photographer here ;)

I took some artistic liberty in choosing color scales for each photo (a) to make them look unique (b) to make people actually look at the color scale (c) because as someone with a physics background it bothers me deeply that people associate hot with red and cold with blue and even go so far as to call 6000K "cool white" and 3000K "warm white" when those are literally blackbody temperature equivalents. So I flipped it around on one shot :)

kbutler2• 6 years ago

Also physics background, and I realized it looked right to me, and had to go back and look at the others...

devenvdev• 6 years ago

I wonder why the church's second floor window is black (fifth photo from the end). Do they insolate floors too? But then why do the walls of the second floor and the roof leak?

simlevesque• 6 years ago

I think that yes they isolate floors too and it looks like the front of the house leaks more heat than then roof.

joshvm2• 6 years ago

(Note: Glass windows are opaque to thermal cameras; the temperature measured here is the surface temperature of the window.)

This is not completely correct. Glass tends to reflect infrared radiation. For example if you view a car windscreen from the air with a thermal camera, you'll see a reflection of the sky.

It's more complicated than that, because you'll also detect the emitted blackbody radiation from the glass (for whatever temperature).

jacquesm3• 6 years ago

Fantastic images, never before did I see a more clear image of what 'continental drift' really looks like.

ChrisArchitect• 6 years ago

(2019) please

https://news.ycombinator.com/item?id=20435516

Show HN: I shot high-res stitched panoramas in Iceland using a thermal camera

Related Stories