Extract Chords From YouTube Videos

http://www.fastcolabs.com/3027777/how-to-extract-sheet-music-from-youtube-videos

by mxpxpx • 12 years ago

83 32 12 years ago

This is so cool!

Makes me wonder what the next step would be. Seems like they're essentially decomposing a song into its ingredients. I wonder if they could use their algorithm to convert a song from one genre to another like auto-creating the hiphop or dance version of a given song :)

ralmeida2• 12 years ago

Actually, they are taking a "big picture" approach, since it's easier to find out the chord produced by an unknown number of instruments than to find out how many instruments are playing and what they are playing, separately. Doing these things you describe would require the second option, I believe.

From the article itself:

“The problem with ‘full polyphonic transcription’ is that the computer doesn’t know how many voices and instruments sound together and what the characteristics are of these instruments,” says De Haas.

“When you transcribe chords, we examine the mixture as a whole and examine what the prominent frequencies are in the spectrum.”

dimmuborgir• 12 years ago

Microsoft Songsmith does that. There are some crazy videos on Youtube. Eg: http://www.youtube.com/watch?v=mg0l7f25bhU

mallin• 12 years ago

This doesn't actually extract sheet music, only the chords present in the music.

notastartup• 12 years ago

This is great, wondering if there's an open source version of something like this.

ralmeida2• 12 years ago

Take a look at http://isophonics.net/nnls-chroma

cannam2• 12 years ago

Which you can run in, for example, http://sonicvisualiser.org/

chillingeffect2• 12 years ago

It's made out of open-source. Their contribution is to integrate it into a web service.

http://chordify.net/pages/about/

http://www.vamp-plugins.org/develop.html

PabloOsinaga• 12 years ago

Melodyne works really well for extracting actual sheet music for polyphonic audio

Here is a demo: http://www.youtube.com/watch?v=jFCjv4_jqAY

It works best with single-instrument signals - not so much with a mix of multiple instruments.

apalmer• 12 years ago

tried to use this and the results were completely off...

BUT i was trying to use it on Moonlight In Vermont by the Johnny Smith Quintet. I am expecting that the type of harmonic structure used in the jazz chord melody style is probably beyond the current capabilities of this technology.

Even still though, i mean the results seemed to have no relation at all to teh music being played, chords were shown on silent parts of the song and vice versa.

Angostura• 12 years ago

Agreed, it' say really cool idea, so I thought I would try it out with something relatively simple - Adele's Skyfall.

The results seem hit and miss - consecutive lines with the same chord structure appear to me transcribed differently

http://chordify.net/chords/adele-skyfall-lyric-video-adeleve...

chillingeffect2• 12 years ago

fwiw, this uses software [1] that has dozens of dials to adjust yet not giving access to those dials. That's probably why. One of the key challenges (i've tried to write chord detectors personally) is that different instruments sounds have different harmonic falloff rates (in different frequency ranges no less).

[1] http://isophonics.net/nnls-chroma

johnvschmitt• 12 years ago

I think you nailed it. Yes, the algorithms can't figure out all the proper tuning yet.

For instance, the lowest frequency (among the strongest amplitude candidates) is often the bass line, & thus the chord. However, those type of general rules are common, yet often broken to make the music more interesting in the first place. Esp. when notes are "left out" to open things up & let you mind fill it in.

hubtree• 12 years ago

I've found the results to be hit and miss, but I find for me it is a good starting off point to figuring a song out.

JonnieCache2• 12 years ago

Harmonic extraction algorithms are cool. Make a recording of yourself standing in a stairwell hitting a wok with a spoon, push that through the algo, and you get avant-garde jazz out the other end. Sometimes. It's certainly easier than spending 30 years learning to play the keyboard.

I guess with it hooked to youtube we can rapidly try out all sorts of unusual inputs.

emersonrsantos2• 12 years ago

Many musicians consider themselves 'composers' because they know harmony. Harmony is 'composition for dummies'. To compose real stuff you have to study counterpoint for lifetime.

And there are even worse algorithmic composing systems like Lyle Murphy's Equal Interval System, Schillinger, etc... I love algorithms but that's not art.

JonnieCache2• 12 years ago

Of course. It's just a fun way of generating raw material and ideas, not for creating finished pieces.

mtinkerhess• 12 years ago

Depends what you mean by "real stuff" I guess. You could just as easily argue that musicians consider themselves composers because they read Fux. Counterpoint is important for certain styles of composition and irrelevant for others.

At its most basic, composition is the creation of a musical score. What form the score takes, what genre the piece is in, the method of composition, the evaluation of the resulting work, are all flexible depending on the style of music and the quirks of the particular composer and audience. Whether something qualifies as "art" is almost entirely subjective -- if it speaks to an audience (including the composer!) it's worth something.

tech-no-logical2• 12 years ago

   this youtube video is not available in your location
   this deezer song is not available in your country
   this youtube video contains content from [x]. it is restricted from playback on certain sites.

on the first three clicks. oh well...

carlob2• 12 years ago

Are you in Germany by any chance?

tech-no-logical2• 12 years ago

close, the netherlands.

cannam2• 12 years ago

You might also like the Yanno chord extractor, with super-academic-looking but quite easy to use UI here: http://yanno.eecs.qmul.ac.uk

If I remember aright, both Chordify and Yanno use NNLS Chroma for the feature extractor but they use different methods to segment and label the chords. (Yanno uses Chordino which you can find at http://isophonics.net/nnls-chroma with the NNLS Chroma plugin.)

emersonrsantos2• 12 years ago

Actual scientific relevant dissertation:

http://dspace.library.uu.nl/bitstream/handle/1874/226713/haa... [pdf]

creamyhorror• 12 years ago

A worthy (free) alternative: CLAM Chordata: http://clam-project.org/wiki/Chordata_tutorial

I've used it before, it's pretty interesting, especially for its visualisations of the chords being played. It works better for songs with a clear chord structure, though. Too bad the project seems kind of dead. There are other projects listed elsewhere in this discussion (e.g. Chordino+NNLS Chroma) which seem viable.

Tangentially, Melodyne's "Direct Note Access" promotional video was very exciting when I saw it years ago, but I have to wonder how well it ever worked (when it finally came out).

baddox2• 12 years ago

I've used Direct Note Access several years ago (soon after it was released), and it's really good. Like creepy good. I put some full-band bluegrass tunes through it and it pretty much caught every note. It doesn't attempt to separate out instruments, so the output was just one MIDI track, but I could easily go in and separate out bass notes, mandolin solos, etc.

It had trouble with most pop and rock music, presumably because of the crushed dynamics and drums. Of course, it's not even designed for analyzing multi-instrument recordings.

carlob2• 12 years ago

I tried it with a song I knew had a very simple chord structure (C A- F G7) and it gave some pretty weird results…

Then I tried it with Don't think twice it's alright and it pretty much nailed it. I wonder if it was tuned on Dylan. :)

jws• 12 years ago

Very nice. I just ran a half dozen varying songs which we perform through it and it mostly matches my chord sheets.

For some bands this will be a dynamic changer. Singers who can not write down chords sometimes don't want to impose on the instrumentalists by bringing in new material. If they can bring in a rough whack that barrier is lifted.

Some notes:

• Not all music is in 4/4. Ok, mixed meter and obscure meter, but 3/4 really ought to be either recognized, or let me tell you. Even when the song is in 4/4, it sometimes loses track of where "1" is.

• It doesn't like to (maybe can't) write more than a 3 note chord, and I didn't see any "sus" chords, but maybe I didn't feed it a good sample. Not understanding 4 note chords will probably add ambiguity to the results, say a Em7 being mistaken for a G if there isn't enough E going on.

• It appears to cache well. When I hit popular songs through their cleanest YouTube URL the results pop up instantly. Well done!

Looking forward to progress, but it goes in the toolbox now.

baddox2• 12 years ago

Curious about the real-world results, I quickly grabbed a link to an artist I know, Andy Zipf, and put his song through Chordify. I played along with the results for most of the song, but wanted to see if the chords were right, directly from the source. I sent Zipf a message on Twitter asking and he responded, “Looks correct.”
http://t.co/aMUnHJiQcP

Perhaps I'm foolish to question the artist about the chords of his own song, but it certainly doesn't look quite right to me. Each phrase in the verse starts on an A, and the algorithm misses that a few times. When the whole band starts, a lot more chord changes get missed, which seems odd to me based on the description of the algorithm.

Also, it would be nice if the tool attempted to identify a tonal center ("key") of the song, and use that to spell the chords more appropriately. For a lot of pop music, this would be fairly easy and reliable. In this song, the tonal center is clearly E, and it would be nice if the C#m wasn't spelled as Dbm.

duskwuff• 12 years ago

Yeah, those chords certainly don't look quite right. There's some missed changes and some that show up a beat too early or too late. It's also missing a lot of subtleties, like 7th chords and inversions.

For a much more blatant example of Chordify just failing to spot really obvious things, check out their results for "Get Lucky":

http://chordify.net/chords/daft-punk-get-lucky-jay-lee

This song has a completely fixed chord scheme (Bm/D/F#m/E). Each exception in the chord sheet is a mistake.

RankingMember• 12 years ago

Perhaps related: There's an app called "Shapes" on iTunes where the devs deconstructed popular songs into "shapes", basically a set of keys that will sound good with the song no matter what you play: http://www.playshapes.com

hesselink• 12 years ago

Chordify is great! Their backend is written in Haskell, and part of it is open source as well: http://hackage.haskell.org/package/HarmTrace

gtani• 12 years ago

they did a couple papers describing algos:

http://ismir2012.ismir.net/event/papers/295_ISMIR_2012.pdf

http://ismir2012.ismir.net/event/papers/LBD2.pdf

(There's usually a few "doesn't work" comments, but imagine how much extra processing is required to get the fundamental when you run guitars thru delay/ reverb, overdrive/distortion, chorus/tremolo/vibrato etc effects)

Extract Chords From YouTube Videos

Related Stories