zebproj

The article makes it sound like this is a very a new idea, but physical models of music instruments, including violin, has been around for over 40 years. Daisy Bell, the first piece of computer music and performed by their model, utilized a physical model of the human singing voice based on measurements of human vocal tract, and that was done in 1962.

Julius Smith wrote pretty comprehensive textbook on the subject of building physical models of musical instruments, available online. Here, for example, is a chapter on modeling bowed string sounds: https://ccrma.stanford.edu/~jos/pasp/Bowed_Strings.html

I often use the general algorithm for 2/1 as my "hello world" when I'm building new generative music systems. You don't need too many ingredients to set it up, and it yields some surprisingly decent sounding results.

The most recent one[0] I made was done when I was playing around with Rust, WASM, and WebAudio. (You'll need to press somewhere to start the sound)

0: https://pbat.ch/isorhythms/

Greetings,

What a beautiful idea. Sadly, I do not think I currently have the skills required to build such a tool.

The underlying algorithms and vocal models I'm using here are just good enough to get some singing vowels working. You'd need a far more complex model to simulate the turbulent airflow required for a cough.

If you suspend disbelief and allow for more abstract sounds, I believe you can craft sounds that have similar emotional impact. A few years ago, I made some non-verbal goblin sounds [0] from very simple synthesizer components and some well-placed control curves. Even though they don't sound realistic, character definitely comes through.

0: https://pbat.ch/gestlings/goblins

Thanks everyone for the suggestions and kind words.

Some details:

The source code for this project can be found on github [0].

I am using an AudioWorklet node with custom DSP using Rust/WebAssembly. Graphics are just done with the Canvas API. The voice leading is done algorithmically using a state machine with some heuristics.

The underlying DSP algorithm is a physical model of the human voice, similar to the model you'd find in Pink Trombone [1], but with some added improvements. The DSP code for that is a small crate [2] I've been working on just for singing synthesizers based on previous work I've done.

0: https://github.com/paulBatchelor/trio

1: https://dood.al/pinktrombone/

2: https://github.com/PaulBatchelor/voxbox

Thanks, I'll look into it

A tutorial would be helpful.

Holding down a note and waiting will cause a second, then a third not to appear. When you move your held, note to another pitch, the other pitches will follow, but with a bit of delay. This produces what is known as staggered voice leading, and produces interesting "in-between" chords.

oh yeah, I made that.

Sporth is a stack-based language I wrote a few years ago. Stack-based languages are a great way to build up sound structures. I highly recommend trying it.

Chorth may need some fixes before it can run again. I haven't looked at it in a while, but I had a lot of fun using when I was in SLOrk.

If you compare codebases, SuperCollider is definitely the more "modern" of the 2. SC is written in a reasonably modern version of C++, and over the years has gone through significant refactoring. Csound is mostly implemented in C, with some of the newer bits written in C++. Many parts of Csound have been virtually untouched since the 90s.

Syntax-wise, Csound very closely resembles the MUSIC-N language used by early computer musicians in the 60s. "Trapped in Convert" by Richard Boulanger was written in Csound in 1979, and to this day is able to run on the latest version of Csound.

Both Csound and SC are both very capable DSP engines, with a good core set of DSP algorithms. You can get a "good" sound out of both if you know what you are doing.

I find people who are more CS-inclined tend to prefer SuperCollider over Csound because it's actually a programming language you can be expressive in. While there have been significant syntax improvements in Csound 6, I'd still call Csound a "text-based synthesizer" rather than a "programming language".

That being said, I also think Csound lends itself to those who have more of a formal background in music. Making an instrument in an Orchestra is just like making a synthesizer patch, and creating events in a Csound score is just like composing notes for an instrument to play.

FWIW, I've never managed to get SuperCollider to stick for me. The orchestra/score paradigm of Csound just seems to fit better with how I think about music. It's also easier to offline render WAV files in Csound, which was quite helpful for me.

You might enjoy my project called sndkit [0]. It's a collection of DSP algorithms implemented in C, written in a literate programming style, and presented inside of a static wiki. There's also a tiny TCL-like scripting language included that allows one to build up patches. This track [1] was made entirely using sndkit.

0: https://pbat.ch/sndkit/

1: https://soundcloud.com/patchlore/synthwave

I actually met BT and asked him about this track.

While it's mostly written in Csound, he "cheated" with the guitar track, which was a recorded sample brought into Csound.

See my other comments here for more info about the underlying technology.

It is pretty incredible that sophisticated digital physical models of the human vocal tract were being done in the early 60s. This was able to be done largely due to the deep pockets of Bell Labs. A lot of R+D was put into the voice and voice transmission.

Many of the simpler vocal tract physical models are very similar to the cascaded allpass filter topologies found in LPC speech synthesizers.

In general, tract physical models have never sounded all that realistic. The one big thing they have going for them is control. Compared to other speech synthesis techniques, they can be quite malleable. Pink Trombone [1] uses a physical model under the hood. While it's not realistic sounding, the interface is quite compelling.

1: https://dood.al/pinktrombone/

Sort of. Both use articulatory synthesis, which attempts to model speech by breaking it up into components and using some coordinated multi-dimensional continuous control to perform phonemes (the articulation aspect). The voder uses analog electronics, while Daisy does it digitally (and without a human performer).

The underlying signal processing used for both is different, but both use a source-filter mechanism.

Does that mean that the similarity in sound to formant-based speech synthesis is because they're both using a sawtooth wave, noise, or other relatively simple sound as the raw input?

Essentially, yes. Both are known as "source-filter" models. A sawtooth, narrow pulse, or impulse wave is a good approximation glottal excitation for the source signal, though many articulatory speech models use a more specialized source model that's analytically derived from real waveforms produce by the glottis. The Lilencrantz-Fant Derivative Glottal Waveform model is the most common, but a few others exist.

In formant synthesis, the formant frequencies are known ahead of time and are explicitly added to the spectrum using some kind of peak filter. With waveguides, those formants are implicitly created based on the shape of the vocal tract (the vocal tract here is approximated as a series of cylindrical tubes with varying diameters).

The singing synthesizer used a surprisingly sophisticated physical model of the human voice [1].

The music was mostly likely created using some variant of MUSIC-N [2], the first computer music language. The syntax and design of Csound[3] was based off of MUSIC-N, and I believe the older Csound opcodes are either ported or based off those found.

Apparently the sources for MUSIC-V (the last major iteration of the MUSIC language) can be found on github [4], though I haven't tried to run it yet.

1: https://ccrma.stanford.edu/~jos/pasp/Singing_Kelly_Lochbaum_...

2: https://en.wikipedia.org/wiki/MUSIC-N

3: https://en.wikipedia.org/wiki/Csound

4: https://github.com/vlazzarini/MUSICV

The neat thing about this particular singing synthesizer is that it used a surprisingly sophisticated (especially for the 60s) physical model of the human vocal tract [1], and was perhaps the first use of physical modeling sound synthesis. Vowel shapes were obtained through physical measurements of an actual vocal tract via x-rays. In this case, they were Russian vowels, but were close enough for English.

While this particular kind of speech synthesis[2] isn't really used anymore, it's still fun to play around with. Pink Trombone [3] is a good example of a fun toy that uses a waveguide physical model, similar to the Kelly-Lochbaum model above. I've adapted some of the DSP in Pink Trombone a few times[4][5][6], and used it in some music[7] and projects[8]of mine.

For more in-depth information about specifically doing singing synthesis (as opposed to general speech synthesis) using waveguide physical models, Perry Cook's Dissertation [9] is still considered to be a seminal work. In the early 2000s, there were a handful of follow-ups to physically-based singing synthesis being done at CCRMA. Hui-Ling Lu's dissertation [10] on glottal source modelling for singing purposes comes to mind.

1: https://ccrma.stanford.edu/~jos/pasp/Singing_Kelly_Lochbaum_...

2: https://en.wikipedia.org/wiki/Articulatory_synthesis

3: https://dood.al/pinktrombone/

4: https://pbat.ch/proj/voc/

5: https://pbat.ch/sndkit/tract/

6: https://pbat.ch/sndkit/glottis/

7: https://soundcloud.com/patchlore/sets/looptober-2021

8: https://pbat.ch/wiki/vocshape/

9: https://www.cs.princeton.edu/~prc/SingingSynth.html

10: https://web.archive.org/web/20080725195347/http://ccrma-www....

From the website:

Note: aubio is not MIT or BSD licensed. Contact the author if you need it in your commercial product.

Module file formats [0] have been around since the 80s, and they can kind of be thought of as MIDI files with sound samples embedded in them. You can get some very impressive sounding full-length tracks in only a few kilobytes.

0: https://en.wikipedia.org/wiki/Module_file

I appreciate the shoutout to Sporth! Admittedly, I haven't used it for quite a few years. But it still works just fine.

In addition to being a part of AudioKit, it also has it's own repository as a self-contained command line program:

https://github.com/PaulBatchelor/Sporth/

I used to use a live-coding setup with Sporth centered around Vim, though it has never been added to the codebase. If anyone is interested in this, please feel free to email me at thisispaulbatchelor at gmail dot com.

These days, I'm so steeped in experimental music, that I was expecting something a bit more "out there" [0]. This feels quite tame by comparison.

I like the idea of a chromatic staff. Making notes visually more evenly spaced would probably make certain things clearer, and probably would provide some interesting insights from a musical analysis point of view. The fact that it still uses the same 5-line staff would make it easier to adopt. It's familiar, and you can use existing staff paper.

However, speaking as a reasonably proficient sight reader, I don't know how much more improvement this adds. From a readability point of view, this looks about the same to me (which I suppose is half the point). Maybe I'm just a jaded musician, but I just don't find music notation that hard to pick up.

Reading music is very similar mental process to reading text. Your brain quickly learns to chunk notes into patterns rather than individual groups, similar to how in reading you don't actually parse out every single letter.

0: https://llllllll.co/t/experimental-music-notation-resources

Csound is a fantastic sound design tool. It was my main composition tool for many years, before I started building my own system. The score/orchestra paradigm is incredibly powerful. It's something I miss quite often in my current system. When I was in music school, I learned how to program by writing programs in Python to generate scores that could then be played by Csound. Scores are so trivial to generate, that you actually don't need to learn that much programming to generating very satisfying algorithmic compositions.

I appreciate that the title of this post calls it a "sound and music computing system" rather than a "musical programming language". In truth, Csound is more of a text-based modular synthesis environment than a programming language.

As others have said, the orchestra syntax is a bit strange at first, but you do get used to it. Writing Csound code feels more like patching a modular synthesizer rather than writing a computer program. It's basically a DSL for connecting small sound/signal modules (called opcodes) together. Most of the time one thinks about things in terms of signal flow and not computer logic. A common mistake I see new Csounders make is to immediately reach for the conditional statements and loops. They often don't behave the way you expect, so people get frustrated.

The Csound dev team has a very strong emphasis on backwards compatibility, to the point where the older opcodes do not get bugfixes in case someone is exploiting the bug in the compositions. The programmer in me groans a little bit, but the composer takes great comfort in the fact that pieces I write now will be playable for many years or even decades (some of the Csound test pieces, like Trapped In Convert by Richard Boulanger or Xanadu by Joeseph Kung, are over 30 years old and still run).

I've been told that many works written in MusicN languages (a precursor to Csound) have been ported to run in Csound, which means that the legacy of Csound includes computer music written in the 60s! I wish I knew where to find those, as I quite enjoy computer music history.

Also a fraction of the DSP capabilities, even with the available third-party plugins (chugins).

Yeah this piece was a huge piece of inspiration for me when I was learning Csound. I actually had a chance to ask BT about this particular piece once and if the rumors were true about it being all done in Csound. The guitar riff you hear halfway through the piece is actually sampled and played via the diskin opcode. Other than that, all Csound. Still a very impressive feat, especially considering the fact that the piece itself was originally mixed in 5.1.

They were great suggestions! Thank you for putting in the time. I hope my responses were helpful.

Good questions. My background is in computer music, and I do use a lot of jargon. A good chunk of them probably come from Csound (and, by extension, the MUSIC N family of languages). Sporth also expects a certain familiarity with modular synthesis, and digital audio systems in general.

what kind of objects are available (generators and filters, I guess, anything more?)

In Sporth, "objects" are referred to as things called unit generators, or "ugens" for short. These things can take in signals as input, and write signals as output. Signals are all audio rate. Even a constant value is an audio-rate signal.

As you mentioned, effects and signal generators are two kinds of unit generators. Another important one are control-signal generator, which include things like low-frequency oscillators (LFOs), or envelopes. These things generally produce signals designed control parameters of other things.

There are also things called ftables (short for function tables I think? another term borrowed from Csound). These things are more or less floating point arrays. Usually they are used for sequences, storing wavetables for table-lookup oscillators, or loading audio samples. An ftable is an ftable, so if you wanted to take a loaded snare sample and use it inside of a sequencer ugen, there's nothing stopping you from that.

what generators are available, and what is their effect

Hard to answer that one concisely. There are approximately 220 unit generators in Sporth right now. Some of them do simple things like addition and multiplication. Others do more complicated things. Some make sound, some process sound, some do a combination of both. The art of composing with Sporth is building an intuition for how these modules will sound together. Admittedly, there is no documentation for all of this in Sporth. There's no time for that. BUT! If you study highly modular computer music systems like Csound or Supercollider, you'll learn concepts that can be applicable to Sporth.

what are triggers. This sentence doesn't explain much "A single trigger is exactly one sample that is a non-zero value (typically, this value is just a '1')"

Ah yes, this is modular synthesizer terminology. If you don't have experience using modular synthesizers, I can see why this is confusing.

Triggers can be thought of as a sort of message, encoded in an audio signal. When a trigger signal occurs, it means "do something". What that something is, depends entirely on the ugen reading the signal. When a trigger signal gets fed into an envelope generator, it tells the envelope to re-trigger the envelope. When a trigger signal gets fed into a sequencer, it means go to the next note in the sequence. Stuff like that.

I just do not understand half of the words in the above sentences.

That was me and the music-dsp mailing list for many years. I've now got around 70-80 percent comprehension there, but yeah jargon. The thing is, once you start memorizing jargon words, you forget what is jargon and what isn't.

I can totally see where you are coming from. It would be great to see Sporth tutorials written for different audiences than mine own. But I need help for that. It's just not a responsibility I am ready to take on. If you do not have the prerequisite background in computer music, there is just so much you need to unpack, and it will never be enough. I'm only one person, and I need what little time I have to try to compose my own music with the tools I write for myself. Selfish by definition, but true.

Very well said. I've come to similar conclusions working with the postfix syntax of Sporth.

Is your language public at all?

They are great as well! Especially Csound. That's where I started. In fact, Sporth uses a lot of DSP code adapted from Csound.

From a language/syntax standpoint, you're still typing way more keystrokes with Csound to do equivalent things in Sporth.

Design philosophies are different as well. Csound is divided up into an orchestra/score, where the orchestra defines the signal flow of an instrument, and the score has instructions on when to play the instruments. Sporth can only define a sound in terms of signal flow, similar to a modular synthesizer. In Csound terms, it's the equivalent of having a single Csound instrument that is always on.

Performance-wise, I've found that Csound has a slight edge, but not by that much. Both are mainly written in C, and both perform quite well in real-time. I use low-end hardware, and I almost never have any issues myself. I wouldn't recommend sporth for embedded systems at all, and to be careful when using it on a raspberry pi 1 (but I said the same thing about Csound too).

From a code infrastructure standpoint, Sporth is way cleaner. And this is mainly what motivated me to write Soundpipe + Sporth, and the biggest strength. It's a very tiny language... the core of it is only a few thousand lines of reasonably C code, with the rest of it being ugen code, which is repetitive and easy to understand. It is built inside of a POSIX environment using simple Makefiles. Sporth code is pretty portable, and can be trivially dropped into a project (AudioKit does exactly this). Csound, on the other hand, is a massive code base full of very cryptic legacy C code, with bits of C++ thrown in for good measure, and uses CMake.

Csound has an amazing collection of opcodes for creating sounds. It's an amazing ecosystem to explore. Sporth has 224 unit generators, which is a lot, but a fraction of what Csound has to offer.

Due to life circumstances, I have been unable to write any real intro documentation.

Compared to Sporth, though, the development model seems less open to community involvement

The biggest reason for this is because I'm actually using Fossil for source control instead of git, and this is a git export for visibility and convenience. Fossil also is built around a cathedral model rather than the bazaar model [0], so it it indeed harder to make one-off commits to the repo. It should be possible to have bidirectional git/fossil support, but I haven't explored it for now.

It would be great to have a "contributing" guide of some sort, or a repo open to pull requests.

There's no guide yet because I'm not ready for any random stranger to be able to make a contribution.

However, if you (or anyone reading) are interested in using developing Patchwerk, please email me at this is paul batchelor at gmail dot com (no spaces).

0: https://www.fossil-scm.org/xfer/doc/trunk/www/fossil-v-git.w...

This sounds really cool, but could use better docs.

I agree. As the creator, it can be hard to write documentation for a beginner audience, so I'd be great to hear your input. What would you suggest?

Right now, here is the documentation that exists now:

Some intro tutorials can be found in the Sporth cookbook:

http://paulbatchelor.github.io/proj/cook/

An interactive version can be found here:

https://audiomasher.org/learn

Once you understand the syntax fundamentals, most of the work is just remembering all the ugens and the argument order. For this, there's a reference guide:

https://github.com/PaulBatchelor/Sporth/blob/master/ugen_ref...

There's a command line utility that comes with sporth called "ugen_lookup" that allows you to look up entries from here:

https://github.com/PaulBatchelor/Sporth/blob/master/util/uge...

Most Sporth ugens are wrappers around Soundpipe modules, for which there is also a reference manual for. It also provides more information about what the module does:

http://paulbatchelor.github.io/res/soundpipe/docs/

it seems like the main advantage is the environment that allows for more fluid iteration combined with functions for audio.

That's exactly right. Having a terse and precise notation for expressing a patch was originally why it was created. There are indeed lower level libraries in other languages powering this (Soundpipe), but I found working in C to be too slow for creative working. Sporth eliminated some of those keystrokes and enabled me to compose music I wouldn't have been able to do otherwise [0].

0: http://paulbatchelor.github.io/sporthlings/