How problematic is resampling audio from 44.1 to 48 kHz?

by brewmarche • 6 months ago

51 113 6 months ago

it's probably worth avoiding the resampling of 44.1 to 48 kHz

Ehhm, yeah, duh? You don't resample unless there is a clear need, and even then you don't upsample and only downsample, and you tell anyone that tries to convince you otherwise to go away and find the original (analog) source, so you can do a proper transfer.

mort969• 6 months ago

If only it was that simple T_T

I'm working on a game. My game stores audio files as 44.1kHz .ogg files. If my game is the only thing playing audio, then great, the system sound mixer can configure the DAC to work in 44.1kHz mode.

But if other software is trying to play 48kHz sound files at the same time? Either my game has to resample from 44.1kHz to 48kHz before sending it to the system, or the system sound mixer needs to resample it to 48kHz, or the system sound mixer needs to resample the other software from 48kHz to 44.1kHz.

Unless I'm missing something?

mort969• 6 months ago

And actually, why do we have both 48kHz and 44.1kHz anyway? If all "consumer grade high quality audio" was in 44.1kHz (or 48kHz) we probably could've avoided resampling in almost all circumstances other than professional audio contexts (or for already low quality audio like 8kHz files). What benefit do we get out of having both 44.1 and 48 that outweighs all the resampling it causes?

zamadatix2• 6 months ago

That's a clear need IMO, but it'd be slightly better if the game could have 48 kHz audo files and downsampled them to 44.1 kHz playback than the other way around (better to downsample than upsample).

zipy1242• 6 months ago

you're not missing something. You can re-sample them safely as stated by the author. They simply state you should check the re-sampler as:

Although this conversion can be done in such a way as to produce no audible errors, it's hard to be sure it actually is.

That is, you should verify the re-sampler you are using or implement yourself in order to be sure it is done correctly, and that with todays hardware it is easily possible.

Veliladon• 6 months ago

Getting pristine resampling is insanely expensive and not worth it.

If you have a mixer at 48KHz you'll get minor quantization noise but if it's compressed already it's not going to do any more damage than compression already has.

ZeroConcerns2• 6 months ago

DAC frequency and the audio format requirements for whatever you supply to your platform audio API have literally nothing to do with each other.

My reply was from an audio mastering perspective.

Joeboy6• 6 months ago

Unless I'm missing something?

I suppose the option you're missing is you could try to get pristine captures of your samples at every possible sample rate you need / want to support on the host system.

zelphirkalt5• 6 months ago

Is this not the job of the operating system or its supporting parts, to deal with audio from various sources? It should not be necessary to inspect the state of the OS your game is running on, to know what kind of audio you can playback. In fact, that could even be considered spying on things you shouldn't. Maybe the OS or its sound system does not abstract that from you and I am wrong about the state of OS in reality, but this seems to me like a pretty big oversight, if true. If I extrapolate from your use-case, then that would mean any application performing any playback of sound, needs to inspect whether something else is running on the system. That seems like a pretty big overreach.

As an example, lets say I change frequency in Audacity and press the play button. Does Audacity now go and inspect, whether anything else on my system is making any sound?

adzm2• 6 months ago

You are right; the system sound mixer should handle all resampling unless you explicitly take exclusive control of the audio device. On Windows at least, this means everything generally gets resampled to 48khz. If you are trying to get the lowest latency possible, this can be an obstacle... on the order of single digit milliseconds.

bob1029• 6 months ago

If 44.1kHz is otherwise sufficient but you have a downstream workflow that is incompatible, there are arguments for doing this. It can be done with no loss in quality.

From an information theory perspective, this is like putting a smaller pipe right through the middle of a bigger one. The channel capacity is the only variable that is changing and we are increasing it.

anonymars• 6 months ago

This isn't the whole picture because they aren't multiples of each other, so there will be interpolation/jitter

For example if you watch a 24fps film on a 60fps screen, in contrast to a 120fps screen

zipy1242• 6 months ago

That seems a rather shallow - and probably incorrect - reading of the source. This is an efficiency and trust trade off as noted:

given sufficient computing resources, we can resample 44.1 kHz to 48 kHz perfectly. No loss, no inaccuracies.

and then further

Your smartphone probably can resample 44.1 kHz to 48 kHz in such a way that the errors are undetectable even in theory, because they are smaller than the noise floor. Proper audio equipment can certainly do so.

That is you don't need the original source to do a proper transfer. The author is simply noting

Although this conversion can be done in such a way as to produce no audible errors, it's hard to be sure it actually is.

That is that re-sampling is not a bad idea in this case because it's going to have any sort of error if done properly, it's just that the Author notes you cannot trust any random given re-sampler to do so.

Therefore if you do need to resample, you can do so without the analog source, as long as you have a re-sampler you can trust, or do it yourself.

sgerenser• 6 months ago

Speaking of a resampler you trust, I’ve had good experience with libsamplerate (http://www.mega-nerd.com/SRC/), which as of 2016 is BSD licensed.

brudgers4• 6 months ago

A very common clear need is incorporating 44.1khz audio sourcesinto video. 48khz is 48khz because 48khz divided by 24fps, 25fps, or 30fps is an integer (and 44.1khz is not).

Also, for decades upsampling on ingest and downsampling on egress has been standard practice for DSP because it reduces audible artifacts from truncation and other rounding techniques.

Finally, most recorded sound does not have an original analog source because of the access digital recording has created…youtube for example.

AshamedCaptain2• 6 months ago

Lots of Live/Audigy era Creative sound cards would resample everything to 48kHz, with probably one of the worst quality resamplers available, to the chagrin of all bitperfect fanatics... still probably one of their best selling sound cards.

I.e. no one cares.

Tanoc• 6 months ago

I had a Soundblaster Live! Gold card back in the day, and I would route my record player or stereo through it so I could use a visualizer on my computer. You could hear the digital noise that was introduced on the highhats. And the source for the sound was a late '70s era Realistic system where everything was analogue. I never knew it was because of the soundcard. I'd always just chalked it up to either Windows XP or VLC doing something.

AshamedCaptain2• 6 months ago

Unlikely to be this issue since this is about resampling of (e.g. from a CD's native) 44.1kHz PCM to 48kHz, and has nothing at all to do when recording since you'd most likely record at 48kHz and play at 48kHz (no HW resampling involved).

everfrustrated• 6 months ago

Changing the sample rate of audio only affects the frequency range. All audio signal is _perfectly_ represented in a digital form.

I am ashamed to admit this took me a long time to properly understand. For further reading I'd recommend:

https://people.xiph.org/~xiphmont/demo/neil-young.html https://www.youtube.com/watch?v=cIQ9IXSUzuM

pxndxx• 6 months ago

Sampling does not lose information below the Nyquist limit, but quantization does introduce errors that can't be fixed. And resampling at a different rate might introduce extra errors, like when you recompress a JPEG.

brudgers4• 6 months ago

In the audio world, quantization is usually discussed in terms of bit-depth rather than sample rate.

dietr1ch• 6 months ago

I see I lose data on the [18kHz..) range, but at the same time as a male I'm not supposed to hear that past in my early 30s, sprinkle concerts on top and make it more like 16kHz :/

At least I don't have tinnitus.

Here's my test,

    ```fish
    set -l sample ~/Music/your_sample_song.flac  # NOTE: Maybe clip a 30s sample beforehand
    set -l borked /tmp/borked.flac # WARN: Will get overwritten (but more likely won't exist yet)

    cp -f $sample $borked

    for i in (seq 10)
        echo "$i: Resampling to 44.1kHz..."
        ffmpeg -i $borked -ar 44100 -y $borked.tmp.flac 2>/dev/null
        mv $borked.tmp.flac $borked

        echo "$i: Resampling to 48kHz..."
        ffmpeg -i /tmp/borked.flac -ar 48000 -y $borked.tmp.flac 2>/dev/null
        mv $borked.tmp.flac $borked
    end

    echo "Playing original $sample"
    ffplay -nodisp -autoexit $sample 2>/dev/null
    echo "Playing borked file $borked"
    ffplay -nodisp -autoexit $borked 2>/dev/null

    echo "Diffing..."
    set -l spec_config 's=2048x1024:start=0:stop=22000:scale=log:legend=1'
    ffmpeg -i $sample -lavfi showspectrumpic=$spec_config /tmp/sample.png -y 2>/dev/null
    ffmpeg -i $borked -lavfi showspectrumpic=$spec_config /tmp/borked.png -y 2>/dev/null

    echo "Spectrograms,"
    ls -l /tmp/*.spec.png
    ```

amluto4• 6 months ago

There’s one thing that bothers me about this. Sure, PCM sampling is a lossless representation of the low frequency portions of a continuous signal. But it is not a latency-free representation. To recover a continuous signal covering the low frequencies (up to 20kHz) from PCM pulses at a sampling frequency f_s (f_s >= 40kHz), you turn each pulse into the appropriate kernel (sinc works and is ideal in a sense, but you probably want to low-pass filter the result as well), and that gives you the decoded signal. But it’s not causal! To recover the signal at time t, you need some pulses from times beyond t. If you’re using the sinc kernel, you need quite a lot of lookahead, because sinc decays very slowly and you don’t want to cut it off until it’s decayed enough.

So if you want to take a continuous (analog) signal, digitize it, then convert back to analog, you are fundamentally adding latency. And if you want to do DSP operations on a digital signal, you also generally add some latency. And the higher the sampling rate, the lower the latency you can achieve, because you can use more compact approximations of sinc that are still good enough below 20kHz.

None of this matters, at least in principle, for audio streaming over the Internet or for a stored library — there is a ton of latency, and up to a few ms extra is irrelevant as long as it’s managed correctly when at synchronizing different devices. But for live sound, or for a potentially long chain of DSP effects, I can easily imagine this making a difference, especially at 44.1ksps.

I don’t work in audio or DSP, and I haven’t extensively experimented. And I haven’t run the numbers. But I suspect that a couple passes of DSP effects or digitization at 44.1ksps may become audible to ordinary humans in terms of added latency if there are multiple different speakers with different effects or if A/V sync is carelessly involved.

Sesse__2• 6 months ago

This is all true, but it is also true for most _other_ filters and effects, too; you always get some added delay. You generally don't have a lot of conversions in your chain, and they are more on the order of 16 samples and such, so the extra delay from chunking/buffering (you never really process sample-by-sample from the sound card, the overhead would be immense) tends to be more significant.

wildzzz• 6 months ago

Wouldn't each sample be just an amplitude(say, 16bit), not a since function? You can't recover frequency data without a significant number of pulses but that's what the low pass filter is for. Digital audio is cool but PCM is just a collection of analog samples. There's no reason why it couldnt be an energy signal.

klaff5• 6 months ago

The xiphmont link is pretty good. Reminded me of the nearly-useless (and growing more so every day) fact that incandescent bulbs not only make some noise, but the noise increases when the bulb is near end of life. I know this from working in an anechoic chamber lit by bare bulbs hanging by cords in the chamber. We would do calibration checks at the start of the day, and sometimes a recording of a silent chamber would be louder than normal and then we'd go in and shut the door and try to figure out which bulb was the loud one.

4gotunameagain• 6 months ago

Does the noise increase when the bulb is near end of life, or does lifespan decrease dramatically when the noise is increased ?

I imagine the noise increases when one of the supports fail, and the filament starts oscillating leading to mechanical stress and failure

(not that it makes a difference, just thinking out loud)

brewmarche4• 6 months ago

This is a nice video. But I’m wondering: do we even need to get back the original signal from the samples? The zero-order hold output actually contains the same audible frequencies doesn’t it? If we only want to listen to it, the stepped wave would be enough then

lucyjojo• 6 months ago

in theory yes, in practice no.

the article explains why.

tldr: formula for regenerating signal at time t uses an infinite amount of samples in the past and future.

hdjxbsksbsh• 6 months ago

All audio signal is _perfectly_ represented in a digital form.

That is not true... A 22kHz signal only has 2 data points for a sinusoidal waveform. Those 2 points could be anywhere I.e you could read 0 both times the waveform is sampled.... See Nyquist theorem.

From memory changing the sample rate can cause other issues with sample aliasing sue to the algorithms used...

temp8830• 6 months ago

Well, yeah, because that's the Nyquist limit at that sample rate. The max frequency needs to be below that (22.999kHz would not be undersampled). Of course, all of these frequencies are in the ultrasound range anyway.

Reducing the sample rate could cause aliasing. Oversampling shouldn't.

comprev• 6 months ago

Thanks for the links very interesting to read.

I buy loads of DJ music on Bandcamp and "downsample" (I think the term is) to 16bit if they only offer 24bit for smaller size and wider compatability.

spacechild1• 6 months ago

That's not downsampling, that's reducing the bit depth. The bit depth determines the resolution of a single sample, i.e. the number of possible values it can represent.

112233• 6 months ago

All audio signal is _perfectly_ represented in a digital form

What? No. All bandwidth limited signal is. Which means periodic. Causal signals like audio can be approximated, with tradeoffs. Such as pre-ringing (look at sinc(x), used to reconstruct sampled signal — how much energy is in the limb preceding the x=0.)

Is the approximation achieved by filtering the 44.1kHz DAC good enough? Yes, yes it is. But the math is way more involved (i.e. beyond me) than simply "Niquist".

This popular myth that limited frequencies we can hear and limited frequencies in Fourier transform sense is the same thing is quite irritating.

cozzyd• 6 months ago

Naively I would upsample by 4-8 by 0 stuffing and low pass filtering and then interpolating. That can't be that bad, can it?

MontagFTB• 6 months ago

We do [cubic curve fitting] all the time in image processing, and it works very well. It would probably work well for audio as well, although it's not used -- not in the same form, anyway -- in these applications.

Is there a reason the solution that "works very well" for images isn't/can't be applied to audio?

amlib3• 6 months ago

AFAIK it introduces harmonic distortion

pixelpoet4• 6 months ago

I'd love to know more about this, do you perhaps have any refs? Thanks

adgjlsfhk16• 6 months ago

The short answer is that our eyes and ears use very different processing mechanisms. Our eyes sense using rods and cones where the distribution of them reflects a spatial distribution of the image. Our ears instead work by performing an analogue forier transform and hearing the frequencies. If you take an image and add lots of very high frequency noise, the result will be almost indistinguishable, but if you do the same for audio it will sound like a complete mess.

bobbylarrybobby• 6 months ago

Where does dither fit into this picture?

mistrial92• 6 months ago

dither is not compression or re-sampling. Dither is a means to change data so that it appears similar to original data to a human AFAIK

source- wrote dithering code for digital images

pixelpoet4• 6 months ago

Dithering is about trading spatial / time resolution for reduced average quantisation error.

Sesse__2• 6 months ago

You generally want to dither when (before) you quantize, unless you have so much headroom that it doesn't matter. E.g., if you're converting 44.1/16 to 48/16 (which involves quantizing each sample from an intermediate higher-precision result), you probably want to dither, but if you're converting 44.1/24 to 48/24, you probably won't need to care since you don't really care about whether your effective result is 24 or 24.5–25 bits.

ErroneousBosh• 6 months ago

That's more of a bit depth rather than bit rate thing. I was surprised to find that going from 16 to 8 bits by simply truncating gave really obvious artifacts on certain sounds (a sampled 808 kick for example had a distinct BZZZEEEOOOOWWWW sound, quite prominent), and even really simple triangular noise dithering made it go away. It did mean the playback was more noisy but it was less obvious.

As an aside, G.711 codecs use a kind of log scale with only four bits of signal but small signal values use much smaller bits.

pixelpoet4• 6 months ago

I'm not sure I understand the "just generate it" perspective. If you want to generate a much higher sampling rate signal that has a common multiple of your input and output sampling rate, "just generating it" is going to involve some kind of interpolation, no? Because you're trying to make data that isn't there.

If you want to change the number of slices of pizza, you can't simply just make 160x more pizza out of thin air.

Personally I'd just do a cubic resample if absolutely required (ideally you don't resample ofc); it's fast and straightforward.

Edit: serves me right for posting, I gotta get off this site.

superjan• 6 months ago

Maybe the following helps: if you have a an analog signal where there are no frequencies above 22.05 khz, it is in principle possible to sample it at 44.1 khz and then perfectly reconstruct the original signal from those samples. You could also represent the same analog signal using 48 khz samples. The key to resampling is not finding a nice looking interpolation, but rather one that corresponds to the original analog signal.

HelloUsername• 6 months ago

(2022)

amlib3• 6 months ago

I wonder if this problem could be "solved" by having some kind of "dual mode" DACs that can accept two streams of audio at different sample rates, likely 44.1khz and 48khz, which are converted to analog in parallel and then mixed back together at the analog output.

Then at the operating system level rather than mixing everything to a single audio stream at a single sample rate you group each stream that is at or a multiple of either 44.1khz or 48khz and then finally sends both streams to this "dual dac", thus eliminating the need to resample any 44.1khz or 48khz stream, or even vastly simplifying the resample of any sample rate that is a multiple of this.

PunchyHamster4• 6 months ago

I wonder if this problem could be "solved" by having some kind of "dual mode" DACs that can accept two streams of audio at different sample rates, likely 44.1khz and 48khz, which are converted to analog in parallel and then mixed back together at the analog output.

You'd just resample both at 192kHz and run it into 192kHz DAC. The "headroom" means you don't need to use the very CPU intensive "perfect" resample.

adzm2• 6 months ago

As a real world example, on Windows, unless you take exclusive access of the audio output device, everything is already resampled to 48khz in the mixer. Well, technically it gets resampled to the default configured device sample rate, but I haven't seen anything other than 48khz in at least a decade if ever. Practically this is a non-issue, though I could understand wanting bit-perfect reproduction of a 44.1 khz source.

klaff5• 6 months ago

I'm kinda shocked that there's no discussion of sinc interpolation and adapting it's theoretical need for infinite signals to some finite kernel length.

For a sampled signal, if you know the sampling satisfied Nyquist (i.e., there was no frequency content above fs/2) then the original signal can be reproduced exactly at any point in time using sinc interpolation. Unfortunately that theoretically requires an infinite length sample, but the kernel can be bounded based on accuracy requirements or other limiting factors (such as the noise which was mentioned). Other interpolation techniques should be viewed as approximations to sinc.

Sinc interpolation is available on most oscilloscopes and is useful when the sample rate is sufficient but not greatly higher than the signal of interest.

oakwhiz• 6 months ago

Sinc interpolation is somewhat resource-intensive / latency inducing which is why it's not used as frequently in realtime audio.

klaff5• 6 months ago

Agree, but it's like knowing a limit imposed by the 2nd law of thermodynamics. You can't do better and if you get close enough you are done.

observationist• 6 months ago

If you're taking something from 44.1 to 48, only 91.875% of the data is real, so 8.125% of the resulting upsampled data is invented. Some of it will correlate with the original, real sound. If you use upsampling functions tuned to features of the audio - style, whether it's music, voice, bird recordings, NYC traffic, known auditorium, etc, you can probably bring the accuracy up by several percent. If the original data already has the optimizations, it'll be closer to 92%.

If it's really good AI upsampling, you might get qualitatively "better" sounding audio than the original but still technically deviates from the original baseline by ~8%. Conversely, there'll be technically "correct" upsampling results with higher overall alignment with the original that can sound awful. There's still a lot to audio processing that's more art than science.

fulafel• 6 months ago

In reality, the amount of precision that can actually be "heard" by the human ear probably lies between 18 and 21 bits; we don't actually know, because it's impossible to test.

This sounds contradictory - what would be the precision that can be heard in a test then?

functionmouse• 6 months ago

This article cleared up so many long-standing questions for me. THANK YOU for sharing!!!

somat• 6 months ago

once we have reduced errors to below the noise floor they are inaudible by definition.

Makes me think of GPS where the signal is below the noise floor. Which still blows my mind, real RF black magic.

nivea3066• 6 months ago

For those looking to delve into this topic more, the term of art is ASRC: Asynchronous Sample Rate Conversion.

How problematic is resampling audio from 44.1 to 48 kHz?

Related Stories