Thousands of bird sounds visualized using Google machine learning

https://aiexperiments.withgoogle.com/bird-sounds/view/

by ptrptr • 9 years ago

232 40 9 years ago

aiexperiments.withgoogle.com

anotheryou2• 9 years ago

more like sorting bird sounds, not visualizing...

The tiny images are just spectograms/fft as far as I can tell.

edit: it's very fun though to click+drag, haha

jboggan• 9 years ago

I really want to map these to a MIDI controller

model_s• 9 years ago

yes, i was expecting there to be a song in the comments already

mannytan• 9 years ago

Stefano D'Alessio was able to make something pretty catchy https://m.youtube.com/watch?v=8hwDD7mYH_k

fairpx• 9 years ago

It's good to see Google in the last couple of weeks launching a bunch of [1]projects that are more in line with their mission of 'organising the world's information'.

[1] http://www.smithsonianmag.com/smart-news/google-digitizes-30...

mortehu• 9 years ago

If you haven't already, try zooming all the way out and drawing things using the grid as a canvas.

mcbits• 9 years ago

Neat, sounds just like my back yard in the morning!

daxfohl• 9 years ago

From the video, it appears this is an AI-plotted hugely multidimensional space t-sne'd onto two dimensions.

Would be interesting to do some kind of ML on how best to present hugely multidimensional spaces onto two _interactive_ dimensions. Where one AI is deciding how things are projected and how it can be manipulated, and another AI is limited to some virtual "mouse, keyboard, 2D screen" to make inferences. Such that it's optimized for faster, more correct inferences.

anotheryou2• 9 years ago

this has to end up in clustering, some color coding for the dimensions would have been nice.

bonoetmalo• 9 years ago

Is there a better way to pan than click-dragging 100 simultaneous bird sounds?

coldcode• 9 years ago

Wonder what it would do with a mockingbird?

jacquesm• 9 years ago

Kill it?

Gigablah• 9 years ago

Run unit tests?

simplehuman• 9 years ago

I see "Oops, sorry for the tech trouble. For the best experience, view in " for chrome on wayland...

jlg232• 9 years ago

Unfortunately they only hint at the envisioned application in the video and don't provide any further links, but the idea is amazing: Use sounds to monitor bio-diversity. Imagine we'd not need cameras and lots of luck to "catch" proof of an animals existence but a grid of interconnected omnidirectional microphones. We'd get real time tracking of individual animals in 3D and could have smartphones literally point the way to yet uncatalogued or even undiscovered species.

edgyswingset• 9 years ago

Somewhat related, but there was a very good talk at MLConf 2017 about using sound to catch illegal logging in the Amazon. Similar premise: collect the sound, analyze the patterns, and classify.

jlg232• 9 years ago

Thanks for the pointer, found the video: https://www.youtube.com/watch?v=VlQyudLKJno

posterboy• 9 years ago

likewise microphones in cities to detect gunshots

zintinio5• 9 years ago

I worked on this as part of a project with US Fish and Wildlife in 2015 for bats! We were monitoring local populations to track the spread of White Nose syndrome. It wasn't in 3D, but we linked the audio data / classifications with lat/long points and plotted population estimates on maps.

pishpash• 9 years ago

Next: add animal languages to Google Translate?

KasianFranks• 9 years ago

Feature request: Play All

mirimir2• 9 years ago

It doesn't work in Firefox. How rude.

natch• 9 years ago

Worked in Firefox for me. However, it was a bit slow to load.

mirimir2• 9 years ago

I got:

Oops, sorry for the tech trouble. For the best experience, view in Chrome browser.

But then, I block some stuff: ads, WebGL, WebRTC, and tracking.

raverbashing• 9 years ago

Works in FF for me, does not work in Chrome

voidmain• 9 years ago

This is my cats' favorite machine learning application so far!

glup3• 9 years ago

Confusing my cat.

sharp11• 9 years ago

As a birder, this looks like a failed experiment to me. Or I don't understand what their goal was. The groupings make little sense in terms of what these species sound like. I'm guessing that's an artifact of the way they sampled the sounds, losing macro properties. Kind of like grouping the words 'paramour', 'enmity' and 'hamster' together bc they all contain /m/ sound.

glup3• 9 years ago

Another birder here, totally agreed. The audio samples are all really short and downsampled.

jaimex2• 9 years ago

What? No Kookaburra?

It has the most unique call of them all.

bkasterm• 9 years ago

There are some instances of the same bird in multiple locations (great horned owl). Presumably multiple recordings of the same bird. My initial reaction to them not being neighboring is to wonder about the quality of the result. Maybe better feature engineering needed to make this biologically relevant. Any other interpretations?

MurrayHill1980• 9 years ago

What task is made easier by this visualization?

tren• 9 years ago

Someone should make a Shazam/Soundhound app for bird calls, I'd definitely buy it if it could narrow it down to a subset of possibilities.

newbear• 9 years ago

Been thinking of this but the library of birds sounds is private to Cornell if I'm not mistaken. The app , at least, should free and open source.

bravura• 9 years ago

I poked around and also looked at a similar experiment, the Infinite Drum Machine: https://aiexperiments.withgoogle.com/drum-machine

Does anyone know what they are doing t-SNE on? i.e. are they just doing t-SNE on the raw waveforms? Or the MFCC spectrogram? Or what?

bane• 9 years ago

It's interesting, at the very local level I think most humans wouldn't think two adjacent bird sounds are all that similar. But if you drag along a long line and listen to a series of different birds you can "hear" a definite organized progression that seems to be organizing rhythm and major tones into groups.

Theodores• 9 years ago

On touch screen it is very BBC Radiophonic Workshop, a nice toy to get some sounds from, to suit a play on Radio 4, that sort of thing. So you could make lots of 'instruments' by applying AI to organising sounds into some space that can be used as per this example and a touchscreen. It my make ornithologists cringe but I think the merit in this work could be elsewhere, as a creative musical instrument of sorts.

glup3• 9 years ago

It would be interesting to see how similarity in bird call behavior tracks (or doesn't) the phylogenetic relationship between species. My hypothesis would be that bird calls are influenced by other birds in the same ecosystem (imitation or differentiation, and reflecting a high degree of cultural learning) rather than the null hypothesis of genetic transmission.

zo1• 9 years ago

If you left-click and drag around on this (with short pauses), you can almost hear something that sounds very close to R2-D2.

thinkMOAR• 9 years ago

Interesting, spammed a few bird lovers i know with it. Though they almost all replied the recordings are not good enough.

Though personally (jk) i was slightly disappointed when i zoomed out i didn't see a big bird (or other bird) likeness.

Thousands of bird sounds visualized using Google machine learning

Related Stories