The Language of Sound, Season Two: the voices of the machine

Watch the season opener: Subtractive. Then Additive, FM, Karplus-Strong, Granular, Wavetable, The Vocal Tract, and the finale, Game Boy. Licensed CC BY 4.0. Season One is complete and live: the playlist.

A sculptor does not add marble. The figure is already in the block, and the work is taking away everything that is not the figure. The oldest trick a synthesizer knows works exactly the same way. That trick is where Season Two begins.

The idea

Season One followed one note all the way up. Waveform to instrument to arrangement to mix to master, and then the machine writing music on its own. Six episodes, a closed loop, finished. You can hand it to someone and say watch this in order and you will get it. The full Season One write-up is here.

Season Two goes somewhere else. Down into the engine room. Every episode is one way a machine makes a voice from nothing: eight methods, eight episodes, each one a complete lesson where the music is built from the very technique it teaches. Same show, same promise. Everything on screen is drawn by the real audio. But it is its own work, not a continuation, and it sounds and looks like one.

Episode one: Subtractive

Subtractive synthesis is the sculptor's approach to sound. You start with too much. A sawtooth wave, a buzzy wave that carries every overtone at once, raw and bright and almost ugly. Then you take a filter, which is just a gate that lets the low part of the sound through and holds the bright part back, the way a wall muffles the party next door. Carve away what you don't want, and what is left is the instrument.

The filter sweep is the star of the episode. You hear the raw saw, then you hear the filter move across it, opening and closing, and the harmonic wall on screen is carved by a curtain that follows the track's real brightness. Add resonance, a small boost right at the filter's edge, and the filter begins to sing as it moves. Give each note its own little sweep, a filter envelope, and the buzz becomes a voice. Oscillator, filter, envelope. That trinity is most of the synthesizer sounds you have ever heard.

The line the whole episode hangs on: you don't build the sound, you reveal it.

Episode two: Additive

Watch Additive.

This one goes the other way. Start with nothing but a sine, the one pure tone with no overtones at all. Then stack more sines on top of it: an octave, a fifth, a third. Each one is just as plain as the first, but together they stop sounding like a test tone and start sounding like an organ, then a choir. You do not carve the sound. You assemble it, one harmonic at a time, and the proportions between them are the whole instrument.

The pipe organ has worked this way for centuries. Every stop you pull adds another rank of pipes, another set of harmonics. It is the oldest synthesizer there is.

On screen a single sine grows partial by partial into a full wave while an organ facade rises behind it, the pipe heights tracing the harmonic series. In the breakdown one pure sine sits beside sixteen stacked together, the same note, and everything between them is harmonics. The score is a warm cathedral in D flat major, every sound in it built from pure sine partials, the opposite mood of Subtractive's dark carve-down.

Episode three: FM

Watch FM.

Subtractive carves a sound down. Additive stacks one up. FM does something stranger. Take a pure tone and let a second wave push its pitch up and down. Slowly, that is just vibrato. Speed the second wave up, hundreds of times a second, and the wobble disappears. New tones appear on either side of the note instead. Those are sidebands.

Then there are only two controls. The index is how hard one wave pushes the other. Push harder and more sidebands pile on, so the sound gets brighter, then brassy. The ratio is the relationship between the two waves. Keep it on a whole number and the new tones land on the harmonic series, so the sound stays musical. Nudge it off a whole number and they land in between, and the note rings like a bell. Push it further and it clangs like metal.

The opening is the lesson played live on one held note: a pure sine, then vibrato, then sidebands, then brighter, then a bell, then metal. On screen, every sideband is drawn where the math puts it, over faint whole-number gridlines, so you watch the ratio move the tones on and off the grid.

John Chowning found this at Stanford in 1967. Yamaha built it into the DX7 in 1983, and it became the sound of a decade. So the score is a DX band, eighties city-pop on the Just the Two of Us changes. Every pitched voice is FM, built like the old factory patches: a six-operator tine electric piano, a feedback bass with a thumb click, swelling brass, FM strings, a breathy lyricon lead, glass bells. In the breakdown two operators make an electric piano, then the DX7's six stack into one bigger voice.

The score took two passes. The first one taught the right thing but did not make the cut. For the second, ChipForge auditioned three directions, and the one that won is called Plastic Night.

Episode four: Karplus-Strong

Watch Karplus-Strong.

This one plucks. It starts with a burst of noise. A click. No pitch at all. Catch that burst in a loop and replay it, over and over, and something strange happens: anything repeated fast enough has a pitch. The noise becomes a note. A harsh one, a buzz carrying every harmonic, ringing forever.

Then the trick that makes it a string. Smooth the loop a little on every pass, averaging each sample with its neighbors. The high harmonics die first and the low ones linger, which is exactly what a real plucked string does. The buzz turns into a guitar, a harp, a koto. Shorten the loop and the pitch goes up. Smooth harder and you get a muted thunk. Smooth gently and it rings.

The June version of this episode skipped that smoothing step, which is the whole reason it works. This one teaches it in order, played live on one held note, and on screen you watch a ring of samples circulate, jagged with the filter off, rounding off pass by pass with it on, while a harmonics panel shows the highs dying first.

Kevin Karplus and Alex Strong published it in 1983, the same year as the DX7. It barely needed a computer: one short memory and an average. In the breakdown, one bare string shows the oldest fact in music. Halve the loop and it jumps an octave. Two thirds of it gives a fifth. A pitch is a length.

The score is melodic house, and every pitched voice in it is that one trick: a koto lead, rolling harp, a tremolo string section, the bass. ChipForge auditioned three directions and the winner is called Paper Lantern. Then it got more strings, because the strings were the best part.

Episode five: Granular

Watch Granular.

This one shatters a sound. Take one sung phrase and chop it into grains, a few hundredths of a second each. Cut them raw and every edge clicks, because each grain starts and stops in the middle of a wave. Give each grain a soft fade in and out and the clicks are gone. Overlap them and the phrase is whole again.

Now scatter them. A few grains a second from random places sounds like rain. Thousands a second is a continuous cloud.

Then the superpower. Every grain plays at its normal speed. Only the read head moves, the place in the sound the grains are taken from. Stop it and one instant holds forever. Move it slowly and time stretches, but the pitch stays exactly where it was. Time and pitch, pulled apart. On screen you watch the actual grains from the score cross a NOW line on the frame you hear them.

Grain size has a floor. In the breakdown the song itself freezes, then the grains shrink from 100 milliseconds to 2.5. Too short to hold a few waves, and the pitch dissolves into clicks. Grow them back and it sings.

Dennis Gabor described sound as tiny quanta in the 1940s. Iannis Xenakis composed with grains. Curtis Roads put it on a computer in the seventies.

The score is future garage, and every pitched voice in it is grains: the bass, the choir, tremolo strings, flicking vocal chops, glass mallets. ChipForge got a new grain engine for it, auditioned three directions, and the winner is called Night Bus. When I heard the raw grains in the reveal I thought they sounded like an old accordion. So now there is one: a folk tune on an accordion built from that same sung "ah", one soft grain on every sixteenth for the reed tremolo.

Episode six: Wavetable

Watch Wavetable.

This one stores a sound. Take a single cycle of a wave, a few thousand numbers. Play it once and it's a tick. Read it round and round and it's a note. Read it faster and the pitch climbs, but the shape stays the same, so the tone does too.

Now store many shapes in a row: a triangle, a square, a saw, a thin pulse, two vowels. That's the table. Slide the read position across it and each shape blends into the next. The tone morphs. The pitch holds still. Every frame is a different recipe of harmonics, so a slide is the spectrum moving, and on screen you watch the score's real frames light up as the music passes through them.

There's a catch. A bright shape read too fast pushes its top harmonics past the highest frequency the sample rate can hold, and they fold back as harsh, out-of-tune tones. That's aliasing. The breakdown plays it, then plays the fix: a smoother copy of the table for every octave.

Wolfgang Palm built it into the PPG Wave around 1980. Today it drives Serum, Vital and a generation of software synths.

This score took three tries. The first blend was pleasant and I said not yet: it sounded less interesting than Game Boy and Granular. The reason was plain once we said it out loud. Nothing in it could only have been made with a wavetable. So the third pass made the moving table position the music itself: Berlin-school sequencers in the PPG tradition, each note kicking its own position, loops of different lengths turning against each other, then locking to the floor. Every pitched sound in it is a wavetable. Only the drums are a stock kit.

Episode seven: The Vocal Tract

Watch The Vocal Tract.

Six machines so far. The seventh is in your throat.

Start with a buzz. Your vocal folds snap shut about a hundred times a second, and each snap is a click. On its own that's a raw drone, every harmonic at once, and no vowel at all. Now send it through a tube: throat, mouth, lips. The tube rings at a few frequencies and lifts the harmonics near them. Those peaks are the formants. Move the lowest two and ahh becomes eee becomes ooo. The buzz never changed.

That's source and filter. The buzz sets the pitch. The shape of the tube sets the vowel. They're independent, and the breakdown plays it in the open: one note with the vowel moving, one vowel with the pitch moving, then breath instead of a buzz, and it whispers.

People have built this for centuries. Kratzenstein's vowel pipes in 1780. A computer at Bell Labs singing Daisy Bell in 1961.

The score is talkbox funk, and every pitched sound in it is a buzz through a vocal-tract filter. Add a burst, a hiss and a hum for the consonants and the same buzz and tube speak. The drop sings the alphabet, A to Z, from mouth shapes measured off a real spoken voice, with our buzz and our melody. A speech recognizer reading the mastered track catches 24 of the 26 letters.

Episode eight: Game Boy

Watch Game Boy.

Seven ways to make a voice. The last one goes back to where this engine began.

1989. A Game Boy. Four sound channels, and that's all. Two are pulse waves, and the duty cycle, how long each wave stays high, sets the tone. At 12.5 percent it's thin and nasal. At 50 it's the hollow square, every even harmonic gone. And 25 and 75 sound exactly the same. One is the other, upside down.

The third channel reads a tiny table, 32 steps of 4 bits, round and round. The fourth is noise from a shift register, a row of bits that feeds itself. A long register hisses. Cut it to seven bits and it repeats so fast it rings, like metal.

Four channels leaves no room for chords. So play the notes fast, one after another, and your ear hears the chord anyway. That's the arpeggio, and it's the sound of a whole era of game music.

ChipForge started life as a Game Boy style four-channel synth, so this one got a real channel engine: band-limited pulses with the four duty cycles, the 4-bit wave channel, 15-bit and 7-bit noise, the volume envelope and the pitch sweep. Every sound in the score is one of those four channel types, the drums included: the kick is the wave channel with a pitch drop, the snare and hats are noise. Chip trance in B minor, a musette accordion made of two thin pulses carrying the hook, the last drop a semitone up. In the middle the channels trade the hook one at a time, and the outro closes on exactly four.

Eight ways to make a voice from numbers. That's Season Two.

Season Two

1. Subtractive, start with everything and carve away. The saw, the filter, the envelope.
2. Additive, build a tone one harmonic at a time and watch the sum become a saw.
3. FM, frequency modulation, one wave bending another into bells, electric pianos, and metal.
4. Karplus-Strong, pluck a string out of nothing but noise and a loop. Koto, harp, a whole string section.
5. Granular, chop a sound into grains and rebuild it as a cloud. Freeze it, stretch it, even make an accordion of it.
6. Wavetable, one cycle of a wave read round and round, sliding through a table of shapes.
7. The Vocal Tract, a buzz through a tube that learns vowels.
8. Game Boy, four little voices, where the whole engine started.

All eight are out. The season is complete. Season Three goes underneath it, into the physics of sound, starting with Air That Moves and The Harmonic Series.

Both seasons, every episode in order with the field guide underneath, are on the series page.

How it is made

Season One was the cool oscilloscope lab, cyan lines on deep ink, the wave drawn in time. Season Two is the warm engine room. Amber, magenta, ember on near-black, and the signature picture moves from time to frequency: spectrum bars, harmonic stacks, the spectrogram running warm along the bottom. That matches what the season teaches. Building a voice means building a spectrum, so you watch the spectrum get built.

The sound world is new too. Nothing reused from Season One. Warmer, darker, analog-leaning, with its own theme in F minor that every episode wears. Der Gouverneur, a deep Alpine philosopher voice, narrates the body of each episode. I host the open and the close, and we both go silent for the drops.

Same machinery as always. Animation in Python and PIL, audio in ChipForge, the numpy synthesizer that is also the subject of the course, narration through ElevenLabs, assembled with FFmpeg. A laptop, no GPU, no subscriptions.

License

Films: CC BY 4.0. Remix it, repost it, drop it into your own thing. Credit "Napkin Films / Organic Arts LLC" and link CC BY 4.0. Engine code (Napkin Films, ChipForge) is GPL-3.0-or-later. ElevenLabs voice audio is licensed content and is not redistributed.