Long distance sound localization with the Raspberry Pi

https://medium.com/@kim_94237/tdoa-sound-localization-with-the-raspberry-pi-3e777469c4fa

by hcfman • 2 years ago

152 40 2 years ago

medium.com

xnx• 2 years ago

Seems to have some similarity to "Finding the Air Cannon": https://news.ycombinator.com/item?id=39188040

hcfman19• 2 years ago

Sorry. My reply to this are in the comments above. Thanks for posting, this is a nice read.

And also points out that the longer the distances involved the less critical time error is making mobile phone time of arrival viable.

hcfman19• 2 years ago

In reference to that "Finding the Air Cannon" article. The author indicated the following facts:

* His listening posts were part of a triangle with sides of length 3.8km, 3.0km and 6.3km

* The vertices were on the roads in Corvallis, Bellfountain Rd, Brooklane Dr and Rivergreen Ave

* The relative times of arrival are 0, 4, 6

* The temperature was 35F or 1 degree celsius

The only way I can make a triangle that fits on those three roads yields a set of the following likely co-ordinates from where his listening posts are:

44.52421,-123.33492 44.53796,-123.29172 44.52975,-123.25571

Adding the relative times to the co-ordinates in a form whereby my localization program can work on it and putting it through my localization program I get the following result:

$ localize_event.sh 1

44.52421,-123.33492 2024-05-27_12-00-00.0

44.53796,-123.29172 2024-05-27_12-00-04.0

44.52975,-123.25571 2024-05-27_12-00-06.0

Enter GPS coordinates and timestamps. Press enter twice to finish. Location: 44.488866130770134,-123.31219694386117

Web links:

OpenStreetMap:

https://www.openstreetmap.org/?mlat=44.488866130770134&mlon=...

Google Maps:

https://www.google.com/maps?q=44.488866130770134,-123.312196...

Which results in a location south and west of the airport as described in the article :)

Knowing how localization works where the sound source is outside of the polygons it could be a bit further South as well.

I'd be very curious to know how close I have been in working this out.

hcfman19• 2 years ago

That’s a nice read, thank you for that. If he sets up three or more recorders then next time he can find out within minutes of the first bang to within meters.

But his article presents a fun challenge. He doesn’t say where his receiver locations are but he does give a triangle dimensions. I wonder if it’s possible to map this onto the street names and then work out which farm it was. It seems likely that there is only one way to map that triangle so each vertex is on the street it is reported to be on.

Then you could invent a time, any time and add the number of seconds that he mentions to get the two other times. Now you have the times and the co-ordinates, within the constraints from the time inaccuracies you could run the docker version of my localization program (https://github.com/hcfman/sbts-aru) on this input and it would output the map link to the offending farm. With a largish degree of error.

Note. From the diagram, the sound source doesn’t fall within the triangle of receivers. This can still work, but the convergence drops off after a time so you can only practically localize this if the distance is not too far outside. But if you make battery powered receivers, you choose a better spot and move one if the recorders so it does fall within the triangle.

Also, just one second of error is a lot!

hcfman19• 2 years ago

Note also, although my project details the construction of a Raspberry Pi recorder for high accuracy localization. If you are localizing over long distances and you are happy a second or two inaccuracy from the arrival times, then indeed you can run the docker version of the program listed on GitHub and perform localizations based on mobile phone determinations of time of arrival. You just have to match the correct input format as detailed in the article. Specifically like the following:

$ docker run -i localize_event 20

Enter GPS coordinates and timestamps. Press enter twice to finish.

51.014131667,5.813736667 2023-09-17_15-49-48.522301

51.015373333,5.811656667 2023-09-17_15-49-48.810330

51.016368332,5.814084879 2023-09-17_15-49-48.710824

51.015238333,5.815941667 2023-09-17_15-49-48.543376

Location: 51.014903961182846,5.814442712946816

Web links:

OpenStreetMap:

https://www.openstreetmap.org/?mlat=51.014903961182846&mlon=...

Google Maps:

https://www.google.com/maps?q=51.014903961182846,5.814442712...

Note also, you can simulate this easily by choosing locations on a map and calculating the expect time of arrivals. Then you can introduce one more seconds of error and see what difference it makes.

It’s lovely to work things out for yourself but he could have saved himself some weeks of sleep by googling sound localization if that was his primary interest :-)

It is weird that you can leg off air cannons all night for three weeks and not have the authorities on your head though.

GJim• 2 years ago

GPS coordinates

*Coordinates, or lat/long

Other navigation systems are available, including other GNSS!

51.014903961182846,5.814442712946816

I'm not sure localisation to the level of the atomic lattice is entirely relevant in this use case.

baq3• 2 years ago

Fun! 1-2m accuracy is surprisingly good (at least for me who has zero experience with such things).

You could probably use something like this in a war zone to locate from where somebody is trying to shoot you. A quick google later yields e.g. https://www.rheinmetall.com/en/products/c4i/reconnaissance-a... or https://transvaro.com/wp-content/uploads/akustik-eng.pdf (which lists accuracy of... 1-2m).

coretx• 2 years ago

Feed the RPI data to https://www.civtak.org/ and send some to Ukraine... Saving lives was never so cheap.

hcfman19• 2 years ago

Sad but true. The world is full of so many places like this :-(

baq3• 2 years ago

The landing page doesn’t describe what TAK is and doesn’t even link to a page which does, even in a hidden menu.

To everyone: if you make something and put it on the web, please spend 10 minutes to write a single paragraph of introduction for people who just learned that your thing exists.

hcfman19• 2 years ago

Interesting. That looks like a system that uses phase difference.

Last weekend I localized an explosion to within 20m when two of the microphones were 3 km apart. I went to the location and they were firing fireworks as part of some religious ceremony.

jupin• 2 years ago

Fascinating. This reminds me of the WWII novel All The Light We Cannot See where the young German soldier is tasked with locating illegal radio transmissions by placing receivers at different places and using trigonometry calculations to estimate the distance and direction of the source. Of course, the culprits would invariably be killed when found, leaving the boy with a deep sense of guilt.

hcfman19• 2 years ago

That was indeed a great movie.

ssfrr• 2 years ago

I did similar long baseline acoustic localization for my dissertation a few years ago. We had about 15 microphones deployed in a wetland for bioacoustic monitoring.

https://resenv.media.mit.edu/pubs/theses/Resynthesizing_Volu...

Rather than manually finding the timestamps in the individual audio files you can do pretty well with using cross-correlation to find the relative delays between the mic signals. Any particular delay between a pair of microphones corresponds to a hyperbola through the mic array. If you visualize the hyperbola from all the pairs of microphones, the correct location pops up as the intersection of them. The cross-correlation peak can get pretty smeared out though, because each microphone is hearing a different version of the source signal.

hcfman19• 2 years ago

Yeah. I want to play with cross correlation next. That could be super interesting.

jkaptur• 2 years ago

I've been toying around (in my head) with the idea of using a diff algorithm to align things like this.

JKCalhoun• 2 years ago

Guessing the problem with the air horn is in part due to how directional it is?

I understand wanting to avoid firing a gun, an explosive.

But my mind recalls putting dry ice in a 2-liter soda bottle containing already hot water. Lid goes on quickly and tightly....

Legal? Not sure. Definitely gives you shotgun-level decibels though. Not at all directional.

thsksbd• 2 years ago

What's wrong with a good ol' fashioned cherry bomb?

spacemanspiff01• 2 years ago

I did this for my senior design project a few years ago.

We had 4 raspberry pis with mic arrays. They were time synced over a wifi mesh network (indoor so no GPS). Each had a detector running, and would send detection segments to a base station to run generalized cross correlation.

It worked pretty well, but we were doing it in room scale 20' environments. Our biggest issue was that we could not disambiguate echos, and had to hack in a hold off period.

We did not end up having time but I wish we could have gotten beamforming working on each pis microphone array, then we could have combined angle of arrival with tdoa, and potentially better handled the echo.

hcfman19• 2 years ago

Indoor and thus smaller distances means time accuracy becomes much more important. In my software chain there is a very small bit of uncortekated latency due to unknown buffering size of the USB driver adding to time inaccuracy potentially.

hcfman19• 2 years ago

Nice! I want to try and get something like that going as well.

Animats2• 2 years ago

That's useful. It wold probably be more useful as a phone app, so that a few people with phones could locate a sound. This is especially useful for low-pitched noise, where human ears are not far enough apart to get any phase difference.

hcfman19• 2 years ago

Over really long distances the time inaccuracy makes less difference so indeed a phone can do interesting things.

I’m a person that likes extremes so the extremely accurate clock time I could get with a GPS synched Oi really appeals to me. Although it’s not really necessary I want to get an rtk GNSS next year to experiment with. Together with ultrasonic microphones I’m pretty sure it could do a great job localizing bats.

Animats2• 2 years ago

Some ham radio friends pointed out that the easy way to synchronize is to relay the sound over a cheap analog radio.

brcmthrowaway• 2 years ago

So this is how city wide gunshot detection works.. nice!!

Simon_ORourke• 2 years ago

I've had an unrealized pet project on my personal to-do list for quite some time in a similar area - to estimate the speed of neighborhood traffic based on the doppler effect of their traffic noise while passing my house. I can't use radar/IR devices since there's I'm not directly adjacent to the road and the intervening land doesn't belong to me, and the substantial hedges make any line-of-sight sensing impractical.

bauruine• 2 years ago

Would this work for music? I assume it's a bit more complicated because it's not clear which bass you are getting at which location. Maybe if you wait for a new track? Has anyone tried something like that?

hcfman19• 2 years ago

Manually it would do. You can use Raven software to locate the booms.

lormayna• 2 years ago

What about having a better estimation of the speed of sound according to temperature and humidity? Do you think it can improve the accuracy ?

hcfman19• 2 years ago

Temperature is already a parameter. Apparently humidity doesn’t have quite such an effect.

In the example further down the page where I attempt to work out where the air cannon was fired from, the “1” is for 1 degree Celsius.

hcfman19• 2 years ago

Oh. I’d like to point out that the localization algorithm itself that I use comes from the lovely opensoundscape project from Tessa Rhinehart and friends from the University of Pittsburg. A big thanks to them for their lovely project.

http://opensoundscape.org

josephernest2• 2 years ago

I tried this too years ago, that was interesting!

How do you time-synchronize the 4 RPi? When you power them off, you lose the sync. So it is difficult to do it correctly.

Also it may be possible that the different RPi have different clock drift over time? i.e. 1.0000 hour on one of them = 1 hour + 20 milliseconds on another one

hcfman19• 2 years ago

My project, sbts-aru,

https://github.com/hcfman/sbts-aru

uses a GPS to synchronise the time with and then even when running completely disconnected from any network the clocks will be accurate to real time with less than 1 microsecond of error. Typically the system time hovers around 100 ns or less from the real time. And I’ve tested this by triggering interrupts on gpio’s on two devices with the same switch and printing the time.

Syncing to sub microsecond accuracy is usually achieved in under a minute.

josephernest2• 2 years ago

If I remember correctly, when you have 2 RPi located at F and F', the solutions M are on an hyperbola : MF = MF' + delta_distance where delta_distance is simply computed with delta_t in the two signals.

hcfman19• 2 years ago

I just use the opensoundscape library that can use as many sound sources as you like in the equation.

Long distance sound localization with the Raspberry Pi

Related Stories