YouTube Video to Tabs and Lyrics

https://github.com/JoinMusic/fish

by sixall • 2 years ago

135 44 2 years ago

github.com

afpx• 2 years ago

Wow - nice job!

riiii• 2 years ago

Very interesting. I don't suppose this would work with instrumental music?

Anyone know of a thing that does?

daniel011• 2 years ago

I tried a piano piece, and it provided some chords, but I'm not sure if the result is correct.

https://lamucal.com/chords/daniel-pal/blue-894793

hayley-patton• 2 years ago

It's better than I would come up quickly with <https://lamucal.ai/songs/synergy/on-presuming-to-be-modern-i> getting the starts, ends and a few of the middle chords in each run, but misses most of the middle chords or gets them wrong.

ndriscoll• 2 years ago

Neat, it would also be awesome to package it into something like a Clone Hero/Rocksmith tool/plugin to generate charts just like Audiosurf did.

CMLab• 2 years ago

AI cover song platforms, How to address issues related to copyright, legal, and ethical concerns?

sixall5• 2 years ago

Thank you for your concern! We take copyright and legal issues very seriously. Our platform employs strict copyright protection measures to ensure that all generated content complies with relevant laws and regulations. We use technical means to prevent unauthorized use and encourage users to respect original works. If you have any copyright-related questions or concerns, please feel free to contact us, and we will address them promptly.

kranner2• 2 years ago

The "tabs" seem to be arpeggiations of the chords, which might have been some use if the chord detection had worked well, which doesn't seem the case. I see chords and tabs being generated from sections which have only spoken audio, while actual guitar parts are not notated at all. The arpeggios are not consistent either and switch arbitrarily to upstrokes/downstrokes and back to arpeggios.

edit: removed a reference to a competing product

LordShredda• 2 years ago

re: edit Can I ask for said competing product? I really like the idea and want to see how others do it

kranner2• 2 years ago

Chordify (web) and Capo (Mac/iOS) but these only do chord detection.

authorfly• 2 years ago

Nice idea, but gives the wrong chords for jazz music: https://lamucal.com/chords/emmet-cohen/after-youve-gone-patr...

E.g. Bb instead of Ebmaj7

Bb7 instead of Bm7b5

guitarlimeo• 2 years ago

I was expecting something like this https://www.youtube.com/watch?v=nLtlyzWuoqM

but was somewhat disappointed. The site is cool, can give you a headstart when transposing a song to practice, but the chords were quite off in a few examples I tried.

liwei0517• 2 years ago

I just trained a Taylor Swift voice model using Lamucal ( https://lamucal.com/ai-cover/share/66be2087bc3fdb000baf3cac ),

The mid-range is eerily close to Taylor's voice - there were moments when I almost thought it was really her singing. But, If you listen closely, you can still catch a tiny hint of that robotic sound.

A Bar Song : (Taylor Swift Cover) https://lamucal.com/ai-cover/song-share/66be244cbc3fdb000e76... Original: A Bar Song (Tipsy) https://www.youtube.com/watch?v=t7bQwwqW-Hc

neilyio• 2 years ago

Very happy to see more tools like this. There is so much potential for interactive tabs and sheet music with YouTube videos.

I only found out about https://www.soundslice.com recently. I'm not sure how it managed to evade me for years of searching for music resources on the internet... but for anyone interested in sheet music, I can't recommend it enough.

The design of the whole platform is so minimal and beautiful, and having notation synchronized with YouTube is simply brilliant. Built by one of the co-creators of Django, too!

adrianh• 2 years ago

Thanks — glad you found our site and like it! :-)

PretzelPirate• 2 years ago

Have you considered adding note detection and note-by-note playing of tabs?

I use that in Rocksmith+, but their detection isn't great and neither is their interface. I'd prefer to use something like Sound Slice with that feature.

TrackerFF• 2 years ago

Seems like a cool concept, but the tab function was more or less useless. Tried a bunch of different songs in various complexities, couldn't get anything convincing.

rrherr2• 2 years ago

Here's the most impressive results I've seen for automated guitar transcription:

High-resolution guitar transcription via domain adaptation

Demo Videos: https://xavriley.github.io/HighResolutionGuitarTranscription... Paper: https://arxiv.org/abs/2402.15258

We propose the use of a high-resolution piano transcription model to train a new guitar transcription model. The resulting model obtains state-of-the-art transcription results on GuitarSet in a zero-shot context, improving on previously published methods.

senbrow• 2 years ago

Anything as good as this for drum tab transcription?

rrherr2• 2 years ago

This isn't exactly what you asked for, but there's a "drumsep" model, which takes a drum audio track and separates it into 6 stems: kick, snare, toms, hi-hat, ride, and crash.

Ctrl+F for "drumsep" in this doc:

Instrumental, vocal & other stems separation & mix/master guide - UVR/MDX/Demucs/GSEP & others - Google Docs https://docs.google.com/document/d/17fjNvJzj8ZGSer7c7OFe_CNf...

bertili• 2 years ago

The MIDI Guitar software [1] does this in real-time, similar to Rolands hexaphonic pickups, but with a standard guitar signal.

MIDI Guitar 3 is in open beta testing [2] and adds MPE (multidimensional polyphonic expression) MIDI.

[1] jamorigin.com [2] jamorigin.com/beta

zerop• 2 years ago

Off topic: what's best way to generate good quality videos given a transcript. Only automation, no manual work. I can code.

bilater• 2 years ago

Try out https://www.shortsgenerator.com

criddell2• 2 years ago

I tried to get it to generate tabs for Where is my Mind by the Pixies. I see the chords, but get the NO icon (red circle with diagonal bar) when I try to click on tabs. Am I doing something wrong?

A couple of weeks ago I asked one of the AIs to teach me this song. It responded that it can't teach specifics or tell me strumming patterns because it would be a copyright violation. I told it that if I went to a human teacher, they would have no problem teaching me how to play along to the song. That was a good enough argument to get the AI to changed its mind (whatever that means) and produced a chord chart and strumming pattern (which was wrong).

thekevan• 2 years ago

Off topic, but you may enjoy this guy's cover and hopefully get some pointers on how to play it as well. Although he definitely plays it differently than most

https://www.youtube.com/watch?v=JhjA2nvVD7U

criddell2• 2 years ago

That's pretty interesting. Thanks for the link.

I was using this video:

https://www.youtube.com/watch?v=ODzhCZ-5f3A

saaaaaam• 2 years ago

It’s a weird technicality of copyright law. When a human teacher plays a song that is a private performance of a composition. When a computer plays a song copyright law treats this as a mechanical copy of the composition and public performance of the composition. When a computer displays musical notation for a composition that is also a mechanical copy.

Private performance of a composition is permitted under copyright law. Creating a mechanical copy and making a public performance requires a licence from a copyright owner and so without that licence there is, in the eyes of copyright law, a breach of copyright.

buildsjets• 2 years ago

Pretty cool! Is there a way to either detect or enforce alternate tunings? There is a world beyond EADGBe... I put in a few songs that I know of which have trivial chord fingerings in drop D, and it comes up with some correct but convoluted chord fingerings in standard tuning.

dadver• 2 years ago

I played around a few minutes with the various features. The voice removing was kind of impressive, though I don't know how novel that is.

I tried making some AI covers, too, which was kind of fun. For one of my tries, I submitted Nirvana's Smells Like Teen Spirit for AI voice generation to make a cover of Carola's song "Främling" (Sweden's ESC song 1983 which came in third, a very non-Nirvana-type of song). At first, I thought the voice sounded pretty much like a Swedish Kurt Cobain, then the more I listened to it, all I could hear was the Swedish artist Nordman, and it dawned upon me that they have similar voice styles. I tried lowering the pitch, and then I was certain I recognised the voice from another artist but couldn't place it. So I'm leaning towards the AI voices being trained on some not-so-unfamiliar artists rather than there being some cool AI magic happening, though I'm out of my depth here.

rwl42• 2 years ago

So I'm trying to understand. Is this spam for the Lamucal service? I saw this same code posted on Reddit the other day under a different name. Here are a few repos with the exact same code under different names:

- https://github.com/DoMusic/Hybrid-Net

- https://github.com/TuneMusic/NiceMusic

- https://github.com/JoinMusic/fish

- https://github.com/Famuse/CombineNet

- https://github.com/AIAudioLab/AITabs

- https://github.com/AIMusicLab/MicroMuisc

I'm pretty sure there are more, but I'll stop there. Especially suspicious considering all the usernames.

Here's a post from yesterday on Reddit:

- https://www.reddit.com/r/coolaitools/comments/1ervthn/found_...

I'm guessing the general process here is:

- Push novelty (but unusable to most people) code to new Github repo

- Submit that code to Reddit/Hacker News

- People see it and are impressed by the novelty code, despite not running it due to missing the models themselves, etc. They upvote and subscribe ($$$) to actually try it.

- Repeat

I understand the desire to promote one's new service, and the product seems like it could be interesting, but this is not the way to get the word out. Reputation matters.

Edit:

Check out the user deeplover's post/comment history. One submission with the MicroMusic (see above) repo, and one comment, see below.

Also, the post by user liwei0517 is almost exactly like BigOrange688 on Reddit. See: https://www.reddit.com/r/MachineLearning/comments/1es0deh/co...

HanClinto• 2 years ago

This comment is a far more interesting story than the project itself.

skeaker• 2 years ago

This is really strange. I'd rather not name names in case I'm wrong, but some of the comments in this thread that link to "things they made" on Lamucal are by very fishy accounts. Account was made a few months ago, only has four comments on seemingly random articles, the comments are either extremely brief or seemingly ChatGPT-generated, then after months of inactivity they pop up in this thread. Not usually one to complain about this but with your comment showing all the other times this has popped up it's way too strange.

sixall5• 2 years ago

I am an entrepreneur, and my startup team is about to fail. In response to the questions raised by rwl4, I am very sorry. All the features on our website are completely free for everyone to use. If we can help some people, it will be a small consolation for the team before the failure. I apologize again.

rwl42• 2 years ago

Look, I get it. Startups are hard. Desperation can lead to poor choices. But spamming isn't the answer and will burn bridges faster than you can build them.

If you are genuinely sorry and want to (possibly) turn this around, here are some ideas:

1. Consolidate your work into one solid well documented Github repo, including a small toy model that people can actually use and play with.

2. Write a blog post or tutorial about your tech. That'll show your expertise without making people feel like they are being duped.

3. Engage in relevant communities genuinely. Build relationships, not just a user base.

4. Use proper channels for promotion... Show HN exists for a reason.

Remember, reputation is currency in this industry. It's hard to earn and easy to lose. Take the long view, build something good, be transparent, and let organic interest do its thing...

If your product is actually solid, there are ways to get it out there. It might be slower, but it's at least sustainable. Good luck.

sixall5• 2 years ago

As an entrepreneur, doing this kind of not-so-glorious thing to promote my own features, I already feel quite pathetic about myself. Being pointed out by rwl4, I can also take a deep breath. I would like to apologize to the people who have been disturbed, and also apologize to the platform.

hu3• 2 years ago

I respect the hustle.

As an entrepreneur, you're light years further than most of us will ever be.

Chin up! Launch again, launch different, launch something else, Pivot if need be. You make me proud.

jokethrowaway• 2 years ago

Cool tech but right now making money with AI is 10x harder than raising from investors.

I'd recommend consulting on the side to keep lights on, there is demand for AI consulting, while there is still hype about AI.

crustaceansoup• 2 years ago

That other, probably-related Hacker News user posted a response to a suicide crisis post that looks like a "four-paragraph standard ChatGPT response", complete with a "Lastly, " at the end. I hope that wasn't done just to increase that account's seeming legitimacy because wow, what incredibly bad taste.

ksr• 2 years ago

I'm looking to do the opposite: Given a melody in MusicXML / MIDI, generate an accompaniment "in the style of". Any pointers?

smrtinsert• 2 years ago

This is going to save me a lot of time for when I actually have time to play music. Much better than whipping out Audacity and its chord estimation, manually grabbing a video etc.

SoftMachine• 2 years ago

I don't often read hackernews for Lamucal AI spams but when I do, it's always nice.

sixall5• 2 years ago

YouTube Video to Tabs and Lyrics

Related Stories