import matplotlib
if not hasattr(matplotlib.RcParams, "_get"):
matplotlib.RcParams._get = dict.get
4.2 Scores vs. timbre#
In Chapter 3, we studied additive synthesis, with a goal of combining harmonics into richer sounds, or timbres (pronounced like the first two syllables of “tambourine”).
Definition 12 (Timbre)
Timbre is the set of attributes of a musical sound that let us recognize it as a distinct musical component or instrument, independent of its pitch and loudness.
Timbre and scores are complementary, equally critical dimensions of how we perceive music. Together, they capture the acoustic and symbolic aspects of music, respectively. The artform of music consists of independently manipulating the two: a score says when and with what parameters, and a timbre says what it sounds like.
As a concrete example, imagine a jazz trio playing a jazz standard notated as sheet music. The three instruments (piano, bass, drums) supply three acoustically distinct timbres, and the sheet music is the score informing which notes they play and when.
A formal view#
Let’s make this a bit more precise. We bring together two ingredients, then define how to combine them.
Definition 13 (Rendering a score)
A score is a set of \(N\) sound events \(\{E_1, E_2, \ldots, E_N\}\), where each event \(E_i = (t_i, \theta_i)\) pairs an onset time \(t_i\) (in seconds) with a dictionary of sound-producing parameters \(\theta_i\).
An instrument (or timbre) is a function \(T_\theta(t) : \mathbb{R} \to \mathbb{R}\) that turns parameters \(\theta\) into a waveform.
Rendering the score produces a single waveform: the sum of every event’s sound, each shifted to its onset time:
The shift \(t - t_i\) places event \(i\)’s sound at its onset \(t_i\). In practice, \(T_\theta(t)\) is nonzero only for a finite window, so on a computer we compute only those nonzero samples and mix them into the output.
Note
This definition is also very general. If \(\theta\) carries a parameter like {"instrument": "bass"}, then a single instrument function \(T_\theta\) can dispatch to entire collections of instruments (e.g., the jazz trio above), choosing how to synthesize each event based on its parameters.
In Pyquist, the method pq.Score.render executes exactly this formula. It takes an instrument as input, i.e., a callable that maps an event’s kwargs to pq.Audio. It then shifts each rendered event to its onset and sums them into a single output pq.Audio. Here is a basic sine-wave instrument and the call that renders our melody:
# An instrument maps one event's kwargs to Audio. Here: an enveloped sine tone.
def sine_instrument(pitch: str, duration: float, f_s: int = 44100, **kwargs) -> pq.Audio:
f_0 = pq.helper.pitch_to_frequency(pq.helper.pitch_name_to_pitch(pitch))
N = int(duration * f_s)
t = np.arange(N) / f_s
tone = np.sin(2 * np.pi * f_0 * t)
result = pq.Audio(tone, f_s)
result *= adenv(0.01, duration - 0.01, N, f_s) # attack/decay so notes don't click
return result
# A simple C-D-E-F-G melody, each note half a second long.
melody = pq.Score([
(0.0, {"pitch": "C4", "duration": 0.5}),
(0.5, {"pitch": "D4", "duration": 0.5}),
(1.0, {"pitch": "E4", "duration": 0.5}),
(1.5, {"pitch": "F4", "duration": 0.5}),
(2.0, {"pitch": "G4", "duration": 0.5}),
])
audio = melody.render(sine_instrument) # render calls the instrument once per event and mixes
pq.play(audio)
The instrument is called once per event, with that event’s kwargs (pitch and duration); render handles the time-shifting and mixing.
Perception: timbre vs. score#
When we studied additive synthesis, we learned that adding harmonics together produces different timbres. In the formal view above, we just learned that adding is also the basis of rendering a score. So where does the dividing line lie between the two?
The dividing line between timbre and score can be surprisingly thin. In Western music, a score is usually a collection of notes, each with a pitch (a fundamental frequency). When several frequencies sound at once, our ear may interpret them as a single unified timbre or as multiple distinct events, depending on the relationships between those frequencies.
We can probe this with two scores. Each has four sound events, played by pure sine tones at different fundamental frequencies, that enter 0.1 seconds apart and then sustain together. The only difference between the two is the set of frequencies:
def osc(f_0: float, N: int, n: int = 0) -> pq.Audio:
t = (n + np.arange(N)) / F_S
return pq.Audio(np.sin(2 * np.pi * f_0 * t), F_S)
# group_a: frequencies that are integer multiples of 220 Hz -> fuse into one timbre
group_a = pq.Score([
(0.0, {"f_0": 220.00, "N": 8.0 * F_S}),
(0.1, {"f_0": 440.00, "N": 7.9 * F_S}),
(0.2, {"f_0": 660.00, "N": 7.8 * F_S}),
(0.3, {"f_0": 880.00, "N": 7.7 * F_S}),
])
audio_a = group_a.render(osc)
# group_b: inharmonic frequencies -> heard as four separate tones
group_b = pq.Score([
(0.0, {"f_0": 220.00, "N": 8.0 * F_S}),
(0.1, {"f_0": 277.18, "N": 7.9 * F_S}),
(0.2, {"f_0": 329.63, "N": 7.8 * F_S}),
(0.3, {"f_0": 392.00, "N": 7.7 * F_S}),
])
audio_b = group_b.render(osc)
pq.play(audio_a, normalize=True) # harmonic (fused)
pq.play(audio_b, normalize=True) # inharmonic (separate)
Tones at 220, 440, 660, 880 Hz
Tones at 220, 277.18, 329.63, 392 Hz
Left (group_a): frequencies that are integer multiples of 220 Hz. Right (group_b): frequencies that are not integer multiples of a common value. Each set runs for eight seconds.
At first, in both examples, you hear four distinct tones as each frequency enters one by one. Over time, however, you may start to perceive the left example as a single, “fused” tone, while the right example continues to sound like four distinct tones occurring simultaneously.
The key distinguishing feature between perceiving timbre or score is whether the frequencies are harmonics of one another (integer multiples of a common fundamental). When they are, as on the left, our ear fuses them into one timbre. When they are not, as on the right, our ear separates them into distinct events. This is precisely why additive synthesis constrains its components to integer multiples of \(f_0\): that constraint is what makes the result sound like a single tone.