ssfrr

At first, I used some random phone chargers, but the Pis would always complain about undervoltage, and throttle their CPU

This is a real annoyance with raspberry pi's. At first you think that you can run them off of USB power, but then you realize that they start throttling at 4.75V, which is still within USB spec (especially when you consider the voltage drop across the USB cable). The point of a spec is that compliant stuff should work, so you shouldn't need a "high-quality" power supply and cable, just compliant ones.

Don't compare your inside to other people's outside.

It's wild that the article frames this as

what started as an experiment in productivity became a runaway success

and

figuring out if the company can afford this level of productivity at scale

It seems like they're equating "developers are spending a ton of money on this" with "this is creating a ton of value".

I'm not saying that AI tools aren't valuable, but the article doesn't question this equivalence at all.

It doesn’t make sense to lump python and Julia together in this high-level/low-level split. Julia is like python if numba were built-in - your code gets jit compiled to native code so you can (for example) write for loops to process an array without the interpreter overhead you get with python.

People have used the same infrastructure to allow you to compile Julia code (with restrictions) into GPU kernels

Is there evidence that minimizing finger movement is ergonomically desirable? It seems like "repetitive" is a key part of RSI, so making the exact same small motion over and over again may not be optimal.

I think about piano players, who obviously need to move their hands and arms a lot to hit the keys (and with more force). Definitely takes a lot more energy than typing on a computer keyboard, but is there evidence that it's any more or less likely to cause injury?

Cool and surprising to see built-in support for the Snyderphonics Manta [1], which is a pretty niche controller. I wrote the `libmanta` library [2] that is vendored into sapf. Haven't touched the library in a few years (though I still use my Manta), so it feels good to see it pop up!

[1] https://snyderphonics.com/manta.htm

[2] https://github.com/ssfrr/libmanta

Thanks for the extra info, I read through some of your entries on GPU optimization and it definitely seems like it's been a journey! Thanks for blazing the trail.

I’m very curious about your experience doing audio on the GPU. What kind of worst-case latency are you able to get? Does it tend to be pretty deterministic or do you need to keep a lot of headroom for occasional latency spikes? Is the latency substantially different between integrated vs discrete GPUs?

Does it work with separate browsers on the same machine? Not sure but I’d guess this sort of filtering would be more common on the browser than the OS

I saw that phrase and thought it was pretty weird. Hunting wild animals for food is not some fringe thing that happens in "other places" I've eaten tons of fish, duck, deer, elk, etc. that were all "wild animals".

I remember back in 2008-ish Johnny Lee at CMU built a cool hack that tracked the user's head using a Wiimote as an infrared camera, and used it for this kind of effect.

https://youtu.be/Jd3-eiid-Uw?t=147

Turns out that head-tracking parallax is surprisingly effective even without stereo vision. I'd guess there's some component about the effect working best when your head motion is large relative to the distance between your eyes, and also best for objects far enough away from your eyes that you're not getting a lot of information from the stereo vision.

I don't know exactly where those thresholds are, but I wouldn't be surprised if a pinball machine is in a regime where it works well.

Say I walk into a machine, and then I walk out, and also an exact duplicate walks out of a nearby chamber. My assumption is that we’d both feel like “me”. One of us would have the experience of walking into the machine and walking out again, and the other would have the experience of walking into the machine and being teleported into the other chamber.

Im probably lacking in imagination, or the relevant background, but I’m having trouble thinking of an alternative.

The underlying clang featurs support compile-time checks as well via the Performance Constraints system: https://conference.audio.dev/session/2024/llvms-real-time-sa...

Doing brute force evaluation on 1024² pixels, the bytecode interpreter takes 5.8 seconds, while the JIT backend takes 182 milliseconds – a 31× speedup!

Note that the speedup is less dramatic with smarter algorithms; brute force doesn't take advantage of interval arithmetic or tape simplification! The optimized rendering implementation in Fidget draws this image in 6 ms using the bytecode interpreter, or 4.6 ms using the JIT backend, so the improvement is only about 25%.

I love how this is focused on how the JIT backend is less important with the algorithmic optimizations, and not on how the algorithmic optimizations give a 1000x improvement with bytecode and 40x with JIT.

it's an unfortunate terminology collision.

- array languages: rank is the dimensionality of an array, i.e. a vector is rank-1, a matrix is rank-2, a N-D array is rank-N

- linear algebra: rank is the number of linearly-independent columns (and also rows)

So for example, if you have a 5x5 matrix where 4 of the columns are linearly independent, it would be rank-4 in the linear algebra sense, and rank-2 in the array language sense.

I guess (though I've never really thought of it before) that you could say that the array-language definition is the rank (in the linear algebra sense) of the index space. Not sure if that's intentional.

OSC (Open Sound Control) is just awesome. It's basically a lightweight protocol on top of UDP packets. It's not hard to roll your own implementation if there isn't one for your platform. It's lacking a lot of features you'd need for a scalable system, but when you just need a few systems to send realtime messages to each other, it's tough to beat.

I've used it a lot for the original designed use-case (sending parameter updates between controllers and music synths), but also a bunch of other things (sending tracking information from a python computer vision script to a Unity scene).

I'm a little confused as to the fundamental problem statement. It seems like the idea is to create a protocol that can connect arbitrary applications to arbitrary resources, which seems underconstrained as a problem to solve.

This level of generality has been attempted before (e.g. RDF and the semantic web, REST, SOAP) and I'm not sure what's fundamentally different about how this problem is framed that makes it more tractable.

zz^* is fine for scalar complex numbers, but z^*z is nice because it also works for vectors. You can think of the complex conjugate as a special case of an adjoint, and the hermetian transpose is another special case.

I did a bunch of contract work last year at a company that was all-in on Julia and it was a really pleasant experience.

IMO one of the issues with Julia is that it’s easy to get nerd-sniped trying to do clever things with the type system and to make as much of your code as possible statically-inferable. Code and libraries that rely heavily on type dispatch ends up throwing MethodErrors deep into the call stack, far away from your code, which makes it harder to debug them.

More mature Julia developers tend to keep things simpler, and make better use of dynamic types instead of contorting to treat it like a statically-typed language.

One of the issues with Julia for this kind of thing is that it’s tuned for throughput more than latency, especially the sort of worst-case latency you worry about for realtime systems. You have to be careful to make sure any methods that are called ahead of time so they’re not JIT-compiled in your audio loop. It’s also hard to write zero-allocation code, which is what you need if you don’t want the GC pausing your program at an inopportune time.

an error of even 1/8 mm in the placement of the camera would result in a useless image.

That doesn’t make sense to me. Presumably part of the image stitching process is aligning the images to each other based on the areas they overlap, so why do they need that much precision in the camera placement? I’d think keeping the camera square to the painting would be important to minimize needing to skew the images, but that doesn’t seem to be what they’re talking about.

The tube amp simulation market is already pretty…saturated. :)

Do these both assume the quantile is stationary, or are they also applicable in tracking a rolling quantile (aka quantile filtering)? Below I gave an algorithm I’ve used for quantile filtering, but that’s a somewhat different problem than streaming single-pass estimation of a stationary quantile.

Here's a simple one I've used before. It's a variation on FAME (Fast Algorithm for Median Estimation) [1].

You keep an estimate for the current quantile value, and then for each element in your stream, you either increment (if the element is greater than your estimate) or decrement (if the element is less than your estimate) by fixed "up -step" and "down-step" amounts. If your increment and decrement steps are equal, you should converge to the median. If you shift the ratio of increment and decrement steps you can estimate any quantile.

For example, say that your increment step is 0.05 and your decrement step is 0.95. When your estimate converges to a steady state, then you must be hitting greater values 95% of the time and lesser values 5% of the time, hence you've estimated the 95th percentile.

The tricky bit is choosing the overall scale of your steps. If your steps are very small relative to the scale of your values, it will converge very slowly and not track changes in the stream. You don't want your steps to be too large because they determine the precision. The FAME algorithm has a step where you shrink your step size when your data value is near your estimate (causing the step size to auto-scale down).

[1] http://www.eng.tau.ac.il/~shavitt/courses/LargeG/streaming-m...

[2] https://stats.stackexchange.com/a/445714

But you'll never be 100% sure. Most musicians aren't willing to pay for NASA-level QA and custom hardware running an RTOS, and even that doesn't guarantee perfect software.

We're always dealing with risk and trade-offs. Maybe you avoid a locking `atomic` synchronization point by implementing a more complicated lock-free ringbuffer, but in the process you introduce some other bug that has you dumping uninitialized memory into the DAC.

I think the advice in TFA is totally reasonable and worth following. I'm just saying that there may be cases where it's OK to violate some of these rules. I'd love to see more data to help inform those decisions.

This isn't even in opposition to the article, which says explicitly:

Some low-level audio libraries such as JACK or CoreAudio use these techniques internally, but you need to be sure you know what you’re doing, that you understand your thread priorities and the exact scheduler behavior on each target operating system (and OS kernel version). Don’t extrapolate or make assumptions

I don't mean to devalue the advice here. I think it's spot on, and I unreservedly recommend this article to folks who want to learn about writing reliable audio software.

I think in essence I'm repeating the comments of Justin from Cockos, which you summarize [1]:

It is basically saying that you can reduce the risk of priority inversion to the point where the probability is too low to worry about.

In that comment you also say:

100% certainty can’t be guaranteed without a hard real-time OS. However 5ms is now considered a relatively high latency setting in pro/prosumer audio circles

Which I interpret as acknowledging that we're already forced into the regime of establishing an acceptable level of risk.

My point is that I would love to see more data on the actual latency distributions we can expect, so that we can make more informed risk assessments. For example, I know that not all `std::atomic` operations are lock-free, but when the critical section is so small, is it really a problem in practice? I want histograms!

[1] http://www.rossbencina.com/code/real-time-audio-programming-...

The context for this article is writing pro audio software, where that kind of distortion would generally be as bad as a dropout, if not worse.

Definitely good to keep in mind. The thing that I think is really interesting about audio programming is that you need to be deterministically fast. If your DSP callback executes in 1ms 99.99% of the time but sometimes takes 10ms, you’re hosed.

I would love to see a modern take on the real-world risk of various operations that are technically nondeterministic. I wouldn’t be surprised if there are cases where the risk of >1ms latency is like 1e-30, and dogmatically following this advice might be overkill.

This is close to the question I asked above, comparing to audio. If you're processing 32 samples at a time at 48kHz, you need to process a frame every 667us. If you only hit the deadline 99.9% of the time, that's a glitch more than once per second. You'd need a bunch more nines for an acceptable commercial product, so most audio devs just treat it as a hard realtime constraint.

I think the main split is whether you care more about average latency or worst-case latency (or some # of nines percentile).

I guess the difference I'm interested in is whether HFT tends to be "realtime", in the sense that there's a hard deadline that you need to hit every time.

Put another way, audio is operating on a much longer timescale, but cares a lot about worst-case latency (as well as throughput and quality). Is that true of HFT also, or are they more concerned with average latency? If computing a trade takes too long, can they just discard it, or is it catastrophic to miss that deadline?