AudioFlux: A C/C++ library for audio and music analysis

https://github.com/libAudioFlux/audioFlux

by CMLab • 2 years ago

230 54 2 years ago

github.com

bravura6• 2 years ago

If this is supposed to be used for deep-learning, shouldn't all the transforms be GPU-accelerated torch functions?

herogary• 2 years ago

Maybe for the convenience of mobile usage?

tgv2• 2 years ago

By the looks of it, those functions extract features (like frequency peaks). You do that once for a sound. The output could function as input for an NN, in which case it would be a tokenizer for sound.

bravura6• 2 years ago

Given what I've seen in audio ML research:

1) Tuning hyperparameters of your audio preprocessing is a pain if it's a preprocessed CPU step. You have to redo preprocessing every time you want to tune your audio feature hyperparams

2) It's quite common to use torchaudio spectrograms, etc. purely because they are faster (I can link to a handful of recent high-impact audio ML github repos if you like)

3) If you use nnAudio, you can actually backprop the STFT or mel filters and tune them if you like. With that said, this is not so commonplace.

4) Sometimes the audio is GENERATED by a GPU. For example, in a neural vocoder, you decode the audio from a mel to a waveform. Then, you compute the loss over the true versus predict audio mel spectrograms. You can't do this with these C++ features. (Again, I can link a handful of recent high-impact audio ML github repos if you like.)

Again, I just don't get it.

aa-jv2• 2 years ago

A GPU is useful, but DSP's are also still useful - for example there is a compelling case to have frameworks around such as AudioFlux, JUCE and others, in order to support portability and also realtime analysis competitively, which is important in this domain, where such things as Qualcomms' ADK, and others, is quite literally being put inside peoples ears...

Not to say that big-AI shouldn't have audio analysis as a compelling sphere of application, but more that, until the chips arrive, in-ear AI is less of a specification/requirement, than in-ear DSP.

We don't need AI to isolate discrete audio components and do things with them, in-Ear. Offline/big-AI, however, is still compelling. But we don't yet have GPU neckbands ..

nesarkvechnep• 2 years ago

What's this C/C++ language?

n4r9• 2 years ago

Some preliminary analysis suggests that if C is an integer greater than 1, C/C++ will always evaluate to 1 [0].

[0] https://www.programiz.com/online-compiler/9fkHTct0Mybpu

gpderetta• 2 years ago

Actually it could be 2 or 1. IIRC the order of evaluation of operator parameters is unspecified (this used to be UB, now is merely unspecified).

/extremelypedantic

troymc• 2 years ago

1/++

rossant• 2 years ago

or 1++ if you got your C operator precedence wrong

iExploder• 2 years ago

mean you are using C++ but only the features that dont suck, so probably 10% of the language, rest is plain C

pjmlp3• 2 years ago

It is the use of English grammar rules to mean C and C++, naturally not everyone was that great on English classes, and specially those that never attended WG21 and WG14 meetings, or work for said companies, and enjoy being pedantic online about it.

To make it easier for those skipping English classes

"A forward dash can be used to state alternatives. A sentence that uses a forward slash in this way can be read to mean that any or all of the stated words could apply."

https://www.thesaurus.com/e/grammar/slash/

And from the world of reference C and C++.

"The C/C++ Users Journal"

https://en.wikipedia.org/wiki/C/C%2B%2B_Users_Journal

"Visual Studio C/C++ IDE and Compiler for Windows"

https://visualstudio.microsoft.com/vs/features/cplusplus/

Random job post from Microsoft,

"Perform software development in C/C++, Python, and other languages."

https://jobs.careers.microsoft.com/global/en/job/1752991/Pri...

Random job post from Apple,

"Develop/maintain bit-accurate function C/C++ model for hardware verification

Develop/maintain cycle-approximate perf C/C++ model for performance analysis - Analyze model

Excellent C/C++ programming skills"

https://jobs.apple.com/en-us/details/200448639/graphics-mode...

Random job post from Google,

"4 years of experience coding with one or more programming languages (e.g., Java, C/C++, Python)"

https://www.google.com/about/careers/applications/jobs/resul...

Random job post from NVidia,

"Strong C/C++ programming skills"

https://nvidia.wd5.myworkdayjobs.com/en-US/NVIDIAExternalCar...

More examples from WG14, WG21 members, C and C++ compiler vendors can be provided.

DragonStrength3• 2 years ago

A library can be written in one or the other. Note most of your "evidence" is job postings, which are generally written by non-technical folks who often mistake Javascript and Java. But no, there is no "C/C++" language or library. There are skills which help you in both. There is code that compiles with both compilers. There is no WG21 for C/C++.

Yes, Visual Studio supports both C and C++, but those are, in fact, two different languages.

You'll struggle to find the links you promised at the end because C and C++ are run by two different groups, meaning you won't be able to link us to single sources.

otabdeveloper4• 2 years ago

It's C when compiled with the gcc or clang compiler.

dsego4• 2 years ago

It's also for Python, I just discovered it a few days ago. This is the website https://audioflux.top/

jcelerier2• 2 years ago

It would be nice to have a comparison with any of the many C++ MIR (music information retrieval) libraries in the wild:

- https://essentia.upf.edu/

- https://github.com/marsyas/marsyas

- https://github.com/ircam-ismm/pipo

- https://github.com/flucoma/flucoma-core/tree/main/include/al...

BrannonKing2• 2 years ago

If a person wanted to transcribe sheet music from recorded audio, do you know which library and features would be the best starting point?

bckr• 2 years ago

Start with source separation using demucs

bravura6• 2 years ago

I have had mixed luck with this model, which is supposedly state-of-the-art: https://github.com/magenta/mt3

What kind of music are you trying to transcribe?

Feel free to email me.

cpdomina• 2 years ago

For MIDI:

https://github.com/Music-and-Culture-Technology-Lab/omnizart and https://basicpitch.spotify.com/

They work better if you apply some source separation before (e.g, https://github.com/sigsep/open-unmix-pytorch, https://github.com/facebookresearch/demucs, or https://mvsep.com)

Still, I think the best results are from proprietary models (specifically https://www.ableton.com/en/manual/converting-audio-to-midi/ and https://www.celemony.com/en/melodyne/what-is-melodyne)

atoav• 2 years ago

Check out librosa as well

Foobar8568• 2 years ago

Personally it's more converting scans of old music sheets to proper ones I am looking for, I haven't seen anything working properly.

dekken_2• 2 years ago

It's C and Python, not C++

dsego4• 2 years ago

C can be used in C++ code, no?

Galanwe• 2 years ago

Something very close, but that's not what you would expect for something that markets itself as a C++ library IMHO. Especially in 2024, most people would hope (or assume) that "C++" means "C++ 11" at least.

Definitely doesn't count as _lying_, but still underwhelming.

dekken_2• 2 years ago

Depends, not all C is C++, eg, there is no (yet) `restrict` keyword in C++ (even if lots of C++ compilers support __restrict__, it's not in the spec)

bregma• 2 years ago

Yes. And C can also be used with Python and Rust. That does not make this a Rust library.

epcoa• 2 years ago

It is true that there is C code that is conforming C++ code. However I would say if you’re using a C compiler with with “extern C” in the headers for C++ linker compatibility (as this library does) then saying C++ is about as misleading as saying a Rust library is C++ as you can link to that too.

As far as compatibility and “history” the languages are different enough now. There are both: features in C that do not exist in C++, and code that is conforming C that would be UB in C++. Saying C/C++ (for real) is usually a dumb target when it’s better to pick one and settle with that.

If it’s C, just say so. Everyone knows what extern C is, you don’t need to confuse.

shultays• 2 years ago

  #ifdef _cplusplus
      #include <iostream>
      #define print() int main(){cout << "Hello world! -- from C++" << endl;}
  #elif (defined __STDC__) || (defined __STDC_VERSION__)
      #include <stdio.h>
      #define print() int main(){printf("Hello world! -- from C\n");}
  #else
  import builtins
  print = lambda : builtins.print("Hello world! -- from Python")
  #endif
  
  print()

Some python code works in C and C++ as well but people don't group them together and call Python/C/C++

textlapse• 2 years ago

All squares are rectangles I guess.

morning-coffee• 2 years ago

Do you have one in safe Rust? See, we've only just met, and I don't know how you handle your ptr/len arguments in C just yet. ;)

gosub100• 2 years ago

Can this be used for audio fingerprinting?

BrannonKing2• 2 years ago

So are they going for feature parity with librosa? I think that would be great.

zombot• 2 years ago

How can a Python library support iOS?

adamnemecek• 2 years ago

It’s a cpp library with python bindings.

AudioFlux: A C/C++ library for audio and music analysis

Related Stories