Generating MIDI melody from lyrics using LSTM-GANs

https://github.com/yy1lab/Lyrics-Conditioned-Neural-Melody-Generation

by groar • 6 years ago

120 25 6 years ago

Judging by the four provided melodies: 1) the notes have very little rhythmic variation. 2) the melodies don't seem to have any concept of metre or metric accent.

thomasfl• 6 years ago

Oh no, they're ruining popular music as we know it. Anybody can just push some buttons and generate the next hit song. All you need is some with artificial lyrics, artificial melody, artificial vocal (Yamaha vocaloid), on top of a beat bought on the net.

narrator• 6 years ago

People idolizing pop stars is something I would be glad to get rid of. Democratize the idols I say. Everyone gets to be a idol!

Juliate3• 6 years ago

Teaching music practice does wonders in this regard. :)

Piskvorrr• 6 years ago

So the next hit song will be indistinguishable from the others, except for the method of generation? Welp...

carlbarrdahl• 6 years ago

When something hard becomes easy with the aid of technology, our expectations and demands of quality tend to rise. Maybe this is a good thing?

What great innovations or paradigms in music has happened recently? For example, what hiphop did in the way it combines beats and poetry revolutionized music as we know it.

nkrisc3• 6 years ago

I know you're being sarcastic but I always thought the concern over the provenance of art is strange. If a computer can generate good music, why does it matter that a computer created it if it's good music? And if computers are just spitting out terrible music, who cares?

Same goes for computer generated visual art, painting created by elephants, or whatever else you can think of that challenges the position of humans as the sole creators of art.

Fnoord• 6 years ago

Supposedly because it devalues the value of manual created artwork, or because people are afraid for change.

And if computers are just spitting out terrible music, who cares?

Well, I loved the 90s goa trance, but it got snowed under by the 00s psytrance. If that 00s psytrance was never made available, perhaps there would've been more of the style I preferred.

My point being that:

1) What is popular matters

A) in ways which might not be apparent at face glance.

B) even if you specifically dislike it.

Juliate3• 6 years ago

Depends if you consider it pure raw material (I think it is) that you can use to convey something of your own or if you can relate to some point to it (and do the same, but to some further extent).

At a rather utilitarian level, one wouldn't consider a spoon exactly the same way whether it was: 1) carved out of wood by someone you can relate to (by history, reasons, knowledge); as a unique piece, or part of a series; 2) or built in a factory, by workers; and available in the same fashion in X instances; 3) or built in a factory by robots; and available as an exact copy in X instances; 4) or a pure unique artefact that got the practical shape of a spoon.

And that would be for a spoon only.

I am of the school of thought that one doesn't make art without the intent of communicating/transmiting (it/through it) with other people, however remote/intimate it can be. If only because one cannot live alone.

In this line, whatever pleasing something can be, if there's no transmission/communication/need/desire behind the creation of it, if there is no social reason behind it, it has no resonance, no affect to me that makes me actually relate to it (and to the person that thought of it, fell on it, carved it in their words, sounds, silences, colors, shapes, steps, etc.).

To me, whatever the artist intent was, when anything is done, it is done by someone - this someone cared, if only enough to make it, and that way attached something to their work.

That is something I can try to relate to (to the product itself, and to the intent perhaps).

When something is produced by a system, there still may be the intent of the person(s) that designed this system. But it may also be totally independant of the designers scope. That is interesting also, as pure rough material.

But it isn't remotely as profound, interesting, human as something that is done by someone real.

archi42• 6 years ago

Why the beat from the net? Just analyze the lyric's measure and produce a beat based on that. I know, it's lame because that doesn't even need ML :P Or, as a drummer, I recommend just going with (1)bumm (2)tzz (3)bumm (4)tz :(goto 1), maybe occasionally inserting a (2+)bumm if you're feeling fancy.

Benchmark proposal: It's good if a significant number out of N=1000 prefer its output for The Raven (by Poe) over Alan Parsons hand-crafted version.

parent seems /s, this is /s.

fit2rule2• 6 years ago

Hello, the 80's called, your moaning about the boring synth musicians interests is deleted.

We synth nerds have been having precisely this argument for decades now, and the answers always, unequivocally is, no matter how you made the music, its the musician, not the tool... as soon as we all agree "computer music is boring", someone proves us wrong.

filoleg• 6 years ago

This. People seem to not realize that there is much more complexity to music besides just the melody and the chord progression used. Which sort of makes sense, because those are probably the only things that people not familiar with making music can easily grasp.

Did the music of the past 10 years do much in those aspects? No, because chord progressions and melodies have been explored into such depths over centuries, it is kinda difficult to do anything new there without sounding ultra experimental and, most likely, not palpable to an average ear. But the music has made giant leaps in terms of sound synthesis, timbre complexity, etc.

Midi generation from lyrics is imo a cool trick, but it doesn't even begin to scratch the surface of "automatic the music-making". It's like saying that using a TypeScript transpiler with syntactic sugar on top is getting us to the point of Javascript code writing itself without need for any software engineers.

matchagaucho• 6 years ago

That's the nature of "pop" music. Too much artificiality one week will spawn an acoustic counterculture movement the next week.

NPMaxwell• 6 years ago

Music, like weaving, is a predecessor in using algorithms to do work. For hundreds of years, music theorists have been codifying the algorithms, or creating new algorithms, that create good music (for particular definitions of good). Like other code instantiated in a network, this code is more tailored to specific prior states and is less available to analysis of its details than prior efforts.

gwern• 6 years ago

Q5 In the case of Markov, etc., I understand that the generation may be not very conformant to the style learnt, unless using a high order Markov but with the risk of recopying entire sequences from the corpus and thus plagiat. But, in the case of a RNN-based architecture [9], what is the rationale?-------------------- A5: As mentioned before, RNN does the similar work as LSTM in our work. But without including a discriminator, it only learns transmission probability between adjacent notes, but does not promise that generated sequences look like real ones.

Come on, guys, that's just not true. You do not need an adversarial loss to get good quality melodies. Look at Sturm's char-RNN on ABC notation, or OpenAI's MuseNet, or a bunch of Project Magenta work, or my own GPT-2 ABC music (MIDI in progress): https://www.gwern.net/GPT-2-music Or for that matter, any generative model trained with a non-adversarial loss (anything using GPT-2 for example).

In fact, generally, everyone avoids GANs for sequence generation because they work so badly compared to regular likelihood training... (Just at a skim, their 'baseline' is pretty suspicious. I'd expect an ablation for the GAN, not comparing their 400-unit LSTM to... a 100-unit LSTM https://www.aclweb.org/anthology/N19-4015.pdf ? Really?)

zozbot234• 6 years ago

feynmanliang's BachBot and hexahedria's biaxial-rnn also have very good melody generation, and these deal with polyphonic music which is quite a bit harder than simple monody/monophonic music.

tabtab• 6 years ago

I always wanted a "demo engine" whereby one feeds in the melody and chord name, and then a style(s). The AI would then use pattern matching to make a fuller score in the chosen style(s). The output could be midi and/or an audio file (such as .WAV). Bonus points for vocals if given lyrics. I could make Elvis diet parodies: "Ain't nothing but a round dog..." Band-in-a-Box software sort of does this, but lacks realism in my opinion.

Generating MIDI melody from lyrics using LSTM-GANs

Related Stories