ArsenArsen

this is just as much of a problem on binary distros mind you. just because there might be a smaller time delta does not mean that the problem is solved

WDYM? 'pragmatic' here means 'not necessarily purely microkernel-y if very difficult'. for instance, the memory manager is in the kernel in managarm

keep in mind that you can't do that, here's a quote that explains it well:

Some programs imprudently rely on code such as the following

    sprintf(buf, "%s some further text", buf);

The glibc implementation of the functions snprintf() and vsnprintf() conforms to the C99 standard, that is, behaves as described above, since glibc 2.1. Until glibc 2.0.6, they would return -1 when the output was truncated.

- printf(3), Linux man-pages 6.04, 2023-04-01

I had anticipated the overhead of XML being brought up, hence the other suggestions there, which is anything + jsonschema. jsonschema was used for illustrative purposes, it's a lot more powerful than most usecases call for, and it pays forward for that by making the syntax very verbose and longwinded. It'd certainly be an alternative with less overhead, though.

I've not benchmarked XML parsing times in a very long time, I'd be interested in seeing the numbers now.

Tabular data is barely data, honestly, hence my PS, I don't think spreadsheets in general are a very good way to store anything.

And yes, absolutely, overengineering is bound to happen, which is unfortunate, but I'm not sure if it really can be avoided while still keeping many of those upsides (especially the rigorous definitions)

I don't think it's much worse than CSV, with it's inflexible structure, not allowing for any formatting to be inserted. As for weird schemas, that really is an implementation specific issue. This example was sketched up in minutes and already reads better than most spreadsheets I've seen over the years!

  <bankstatement>
      <transaction date="2021-08-19" description="Hello, world!">
          <debit commodity="USD">100</debit>
          <balance commodity="USD">237.87</debit>
      </transaction>
  </bankstatement>

AFAICT, XML fulfils all of the requirements of this.

It is self describing, rigorously defined for the machine, highly flexible, highly extensible, human readable, text-based and an open format, and it is only mentioned as a storage format in the post, not even nearly doing it justice!

XML is complex, but there's already very established libraries for handling it, so that shouldn't be an issue, and that's the only drawback XML has.

Hell, even alternatives like JSON/YAML + JSONSchema exist, still providing incredibly rigorous validation, but staying human readable and ubiquitous.

P.S. I think a bigger failure here is trying to generalize the concept of spreadsheets, rather than using a common encoding (XML) for domain-specific data formatting.

TeamSpeak has cemented Mumbles paradigm in the minds of many, many mainstream people years before Discord existed. Mumble works pretty much the same in terms of convenience and simplicity. It's not that niche. Try it before you knock it.

They also support RNNoise, and everyone who knows me knows I love RNNoise! :)

I'm pretty sure the switch you're talking about is hardware level. If you want to turn rnnoise on/off you could create a new pcm rather than overriding the default, and then having the software select it. The README example of asoundrcs is for the most part the same, you'd just remove this secion:

  pcm.!default {
      type asym
      playback.pcm "cards.pcm.default"
      capture.pcm "rnnoise"
  }

  pcm.!default {
   type asym
   playback.pcm "plug:jack"
   capture.pcm "plug:rnnjack"
  }
  
  pcm.rnnjack {
   type rnnoise
   slave.pcm plug:jack
  }

There's no GUI for it, but I'm willing to help you with it. If you have no asoundrc (i.e. you let ALSA figure it out for you) the example in the README will work. Otherwise you can email me or post on the mailing list and I can get back to you.

Coming back with information from #xiph on freenode:

  16:57 <ArsenArsen> where and under what license is the training data used for RNNoise?
  18:38 <rillian> ArsenArsen: There's a copy of what I believe is the training data on the xiph server, but afaik it's never been published
  18:39 <rillian> the original submission page has an EULA waiving copyright and liability claims, and agreeing that it _may_ be released CC0.
  18:40 <rillian> it looks like that didn't actually happen.
  18:41 <rillian> there may have been concerns about auditing it for privacy issues, but there's a lot of audio to listen to, 6.5G compressed
  18:41 <rillian> jmspeex, TD-Linux: what's the status of publishing the rnnoise training data?
  18:43 <jmspeex> Are you talking about the data that was used to train the default RNNoise model or the noise that got collected with the demo?
  18:43 <rillian> jmspeex: I think debian just cares about the training data for the default model.
  18:44 <jmspeex> There was never plan to release that -- it includes data from databases we cannot release
  18:44 <jmspeex> but I don't see what the issue is. Distributing the model is not the same as distributing the data
  18:45 <rillian> ah, I see. I didn't realize you'd used proprietary sources as well.

Right now there's no such feature in place, but I imagine keeping the buffer from before denoising and mixing it into the denoised result (plus the multiplication) will do what you're describing? It may increase volume, I might need to reduce the volume of the denoised audio first. I'll play around with it, and am open to hearing what you've got to say about it.

There's an issue tracker on that page, under tickets, but I'd prefer if you took a discussion to the attached mailing list first before it hits the official tracker.

As for packaging, that's my field of work for some projects I'm working on so it's not unfamiliar to me, the only problem is that the RNNoise upstream lacks releases, although there's discussion about something happening about that.

I'd be quite interested in such an article, again, my goal (besides VoIP) is screencasting and/or streaming, so any bit of advice someone with experience might have is greatly useful.

I'll look into expansion and compression, and I could implement a wet/dry setting that multiplies the source samples and then mixes them into the result, if I understood the concept right.

EDIT: RNNoise seems to be alright when it comes to canceling noise during speech too, I didn't notice it overdoing it.

+1, that data is CC0, and I believe that's all the data that was used for training.

It is possible to use VST2 on Windows. This way you get RNNoise and the advantages of Free software.

https://github.com/werman/noise-suppression-for-voice

There is training instructions in the repository. The training scripts appear to be using some pretty standard ML libraries (I'm seeing keras and mentions of tensorflow), so I imagine that the requirements are the same as those.

I don't feel I'm qualified to elaborate on this specifically, again, I'm no ML person. For more info look here: https://github.com/xiph/rnnoise/tree/master/training https://github.com/xiph/rnnoise/blob/master/TRAINING-README

I believe Pulse also has a LADSPA module that you could try.

The plugin uses very little CPU, and is entirely inactive when not in use (i.e. when data isn't being pulled => the transfer function isn't being called) due to how ALSA works

EDIT: Do note, though, that each process pulling audio will be denoising independently, so the usage scales linearly with the amount of clients. This is due to how ALSA plugins work, but regardless of that, on a Ryzen 5 1600x (the only CPU I can test on), the plugin uses 2.5% of a single core when recording mono 48k

The documentation is a bit poor. The original data is available for download (with more info about the entire process, most of which is outside of my grasp as I am not an ML person) in the demo blog post: https://jmvalin.ca/demo/rnnoise/ (towards the bottom of the page)

Ever since I got a new microphone I've been having issues with large quantities of background noise, primarily typing sounds, leaking in through my microphone. A friend of mine had told me about rnnoise, describing it as "very good" at removing noise, so I decided to test it. My initial testing started by me piping raw PCM from arecord through rnnoises denoiser demo into aplay. The results frankly shocked me. When not speaking, there was no noise (or sound) at all (this is due to rnnoises voice detection system, which essentially mutes the microphone when there's no voice), and when I was talking, the sound of my keyboard was a lot quieter without it affecting my voice, this lead me to decide to develop alsa_rnnoise to have good real time noise cancellation.

alsa_rnnoise is a very simple ALSA filter plugin that runs its input through rnnoise before outputting it back to ALSA. It operates in real time, adds very little latency (the amount depends on what size frames ALSA delivers, but is nominally less than 10ms). After enabling it, any annoying background sounds were gone. My intended use cases for this were VoIP, screencasting and streaming, and, as far as I can tell, it works great for all three.

Ever since I got a new microphone I've been having issues with large quantities of background noise, primarily typing sounds, leaking in through my microphone. A friend of mine had told me about rnnoise, describing it as "very good" at removing noise, so I decided to test it. My initial testing started by me piping raw PCM from arecord through rnnoises denoiser demo into aplay. The results frankly shocked me. When not speaking, there was no noise (or sound) at all (this is due to rnnoises voice detection system, which essentially mutes the microphone when there's no voice), and when I was talking, the sound of my keyboard was a lot quieter without it affecting my voice, this lead me to decide to develop alsa_rnnoise to have good real time noise cancellation.

alsa_rnnoise is a very simple ALSA filter plugin that runs its input through rnnoise before outputting it back to ALSA. It operates in real time, adds very little latency (the amount depends on what size frames ALSA delivers, but is nominally less than 10ms). After enabling it, any annoying background sounds were gone. My intended use cases for this were VoIP, screencasting and streaming, and, as far as I can tell, it works great for all three.