import matplotlib
if not hasattr(matplotlib.RcParams, "_get"):
    matplotlib.RcParams._get = dict.get

11.0 Sampling

We first define precisely what we mean by a sample: a contiguous chunk of audio samples extracted from some source recording.

Definition 33 (Sample)

Let \(\mathbf{x} = [x[0], x[1], \ldots, x[N-1]]\) be source digital audio at sample rate \(f_s\) of length \(N\) samples. A sample of \(\mathbf{x}\) is a contiguous chunk

\[[~x[m], x[m+1], \ldots, x[m+M-1]~]\]

of length \(M \leq N\) audio samples, starting at offset \(m \in \{0, 1, \ldots, N - M\}\).

In units of seconds, the sample starts \(\frac{m}{f_s}\) seconds into the recording and lasts \(\frac{M}{f_s}\) seconds.

We have seen operations like this before. When we extracted frames for frame-based processing and granular synthesis in Chapter 10, each frame was also a contiguous chunk of audio samples.

The key difference here is how we choose \(m\) and \(M\). In frame-based processing, they are determined procedurally: the offsets are multiples of the hop length \(N_H\), and every frame has the same frame length \(N_F\). When sampling, by contrast, a person chooses \(m\) and \(M\) (the start time and duration of the sample) to capture something musically meaningful, like a single drum hit or a memorable phrase. They typically make this choice by ear, or visually by looking at the waveform.

The Amen break

The Amen break is a seven-second drum solo, played by Gregory C. Coleman on The Winstons - Amen, Brother (1969). Starting with hip-hop in the late 1980s, producers sampled it relentlessly, and it became the rhythmic backbone of entire genres like jungle and drum and bass. It is often called the most sampled recording in history, appearing in thousands of tracks.

Try it yourself below, using one of the most famous samples of all time: the Amen break. Choose an offset \(\frac{m}{f_s}\) and a duration \(\frac{M}{f_s}\) (both in seconds), and use the zoom to inspect the waveform up close. Can you isolate a single snare drum or cymbal hit?

# hide
# no-output
from IPython.utils.capture import capture_output
with capture_output():
    %pip install -q plotly anywidget

import asyncio
import os
import numpy as np
import plotly.graph_objects as go
from plotly.subplots import make_subplots
import ipywidgets as widgets
from IPython.display import Audio
import pyquist as pq
import icm_plotly
from icm_plotly import RED, BLUE, GOLD, IRON, STEEL

Drag the sliders to choose a sample: its offset \(m / f_s\) and its duration \(M / f_s\), both in seconds. The sample is highlighted in blue, and the zoom slider magnifies the waveform around it. The audio card underneath always plays the current sample.

# hide
# autorun
amen = pq.Audio.from_file("./assets/amen.wav").as_mono()
sr = amen.sample_rate
x = np.asarray(amen.samples).reshape(-1)
T = len(x) / sr
YMAX = 1.1 * np.abs(x).max()
OFFSET0, DURATION0, ZOOM0 = 1.744, 0.436, 1.0   # the first beat of the second bar


def view(lo, hi, x=x, sr=sr, max_pts=2000):
    # times and values of x[lo:hi], min/max-decimated so the peaks survive
    lo, hi = max(0, int(lo)), min(len(x), int(hi))
    seg = x[lo:hi]
    step = max(1, len(seg) // (max_pts // 2))
    n = len(seg) // step * step
    if step == 1 or n == 0:
        return (lo + np.arange(len(seg))) / sr, seg
    blocks = seg[:n].reshape(-1, step)
    t = (lo + np.arange(0, n, step) + step / 2) / sr
    return np.repeat(t, 2), np.column_stack([blocks.min(1), blocks.max(1)]).reshape(-1)


def window(offset, duration, zoom, T=T):
    # the visible time range: T / zoom wide, centered on the sample (when
    # zoomed in far enough, the sample can extend past both edges)
    width = T / zoom
    lo = min(max(0.0, offset + duration / 2 - width / 2), T - width)
    return lo, lo + width


def figure():
    lo, hi = window(OFFSET0, DURATION0, ZOOM0)
    a, b = OFFSET0, OFFSET0 + DURATION0
    t_all, y_all = view(lo * sr, hi * sr)
    t_seg, y_seg = view(a * sr, b * sr)
    fig = go.Figure()
    fig.add_scatter(x=[a, a, b, b, a], y=[-YMAX, YMAX, YMAX, -YMAX, -YMAX],
                    fill="toself", fillcolor="rgba(0, 123, 192, 0.12)",
                    line=dict(width=0), hoverinfo="skip")
    fig.add_scatter(x=t_all, y=y_all, mode="lines", line=dict(color=IRON, width=1))
    fig.add_scatter(x=t_seg, y=y_seg, mode="lines", line=dict(color=BLUE, width=1))
    fig.update_xaxes(range=[lo, hi], title_text="Time (s)", fixedrange=True)
    fig.update_yaxes(range=[-YMAX, YMAX], title_text="Amplitude", fixedrange=True)
    fig.update_layout(showlegend=False)
    return fig


def controls(fig):
    offset = widgets.FloatSlider(description="Offset (s)", min=0.0, max=round(T - 0.01, 3),
                                 step=0.001, value=OFFSET0, readout_format=".3f")
    duration = widgets.FloatSlider(description="Duration (s)", min=0.01, max=round(T, 3),
                                   step=0.001, value=DURATION0, readout_format=".3f")
    zoom = widgets.FloatSlider(description="Zoom", min=1.0, max=40.0, step=0.5,
                               value=ZOOM0, readout_format=".1f")
    readout = widgets.HTML()

    # the defaults snapshot the arrays; the page's notebooks share one kernel
    def update(off, dur, z, view=view, window=window, sr=sr, T=T, N=len(x)):
        dur = min(dur, T - off)                  # a sample can't run past the end
        lo, hi = window(off, dur, z)
        t_all, y_all = view(lo * sr, hi * sr)
        t_seg, y_seg = view(off * sr, (off + dur) * sr)
        with fig.batch_update():
            fig.data[0].x = [off, off, off + dur, off + dur, off]
            fig.data[1].x, fig.data[1].y = t_all, y_all
            fig.data[2].x, fig.data[2].y = t_seg, y_seg
            fig.layout.xaxis.range = [lo, hi]
        readout.value = (f"<span style='font-size:0.9em'><i>m</i> = {int(off * sr):,} "
                         f"&nbsp;·&nbsp; <i>M</i> = {int(dur * sr):,} &nbsp;·&nbsp; "
                         f"<i>N</i> = {N:,} samples</span>")

    widgets.interactive_output(update, {"off": offset, "dur": duration, "z": zoom})

    # the audio card under the controls: the previous clip stays in place
    # while you drag (so the layout never jumps) and is swapped for the new
    # one when the pointer releases (keyboard nudges settle on a timer). It is
    # written through the Output's synced `outputs` trait, which works
    # outside a kernel message, where display() output has no destination
    out = widgets.Output()
    gate = icm_plotly.release_gate()   # pointer state: is a slider mid-drag?
    pending = []
    dirty = []

    def render(x=x, sr=sr, T=T):
        off = offset.value
        dur = min(duration.value, T - off)
        y = x[int(off * sr):int(off * sr) + int(dur * sr)].copy()
        n = min(len(y) // 2, int(0.002 * sr))      # 2 ms fades so the cuts don't click
        y[:n] *= np.linspace(0, 1, n)
        y[len(y) - n:] *= np.linspace(1, 0, n)
        y *= 0.125 / max(np.abs(y).max(), 1e-6)    # about -18 dBFS, a safe level
        audio = Audio(y.astype(np.float32), rate=sr, normalize=False)
        data, metadata = get_ipython().display_formatter.format(audio)
        # one assignment swaps the old card for the new one in place, so
        # the page never shows an empty card and nothing shifts
        out.outputs = ({"output_type": "display_data",
                        "data": data, "metadata": metadata},)

    async def settle():
        await asyncio.sleep(0.25)
        pending.clear()
        if dirty and not gate.dragging:
            dirty.clear()
            render()

    def on_change(_):
        dirty.append(True)
        if pending:
            pending.pop().cancel()
        pending.append(asyncio.ensure_future(settle()))

    def on_release(change):
        if not change["new"] and dirty:
            if pending:
                pending.pop().cancel()
            dirty.clear()
            render()

    gate.observe(on_release, names="dragging")

    for s in (offset, duration):                   # zooming doesn't change the sound
        s.observe(on_change, names="value")
    if not os.environ.get("ICM_BOOK_BUILD"):   # the build bakes no card
        render()
    return widgets.VBox([offset, duration, zoom, readout, out, gate])

icm_plotly.show(figure, controls)

Offset (s)1.744

Duration (s)0.436

Zoom1.0

m = 76,910  ·  M = 19,227  ·  N = 307,411 samples

Extracting samples in Pyquist

In pyquist, extracting a sample is a one-liner. We can work in units of seconds with pq.Audio.segment, whose offset and duration arguments correspond to \(\frac{m}{f_s}\) and \(\frac{M}{f_s}\). Or we can work in units of audio samples by slicing an pq.Audio directly with audio[m:m + M], just like a NumPy array.

Below, we extract the second bar of the Amen break both ways. Each beat of the break lasts about \(0.435\) seconds, so the second bar starts \(4 \times 0.435 = 1.74\) seconds into the recording and lasts just as long:

# hide
import numpy as np
import pyquist as pq

amen = pq.Audio.from_file("./assets/amen.wav")
f_s = amen.sample_rate
print(f"N = {len(amen):,} samples ({amen.duration:.2f} seconds at {f_s} Hz)")
pq.play(amen)
pq.plot(amen)

N = 307,411 samples (6.97 seconds at 44100 Hz)

<Axes: xlabel='Time (s)', ylabel='Amplitude'>

beat = 0.435  # approximately the duration of one beat of the break
offset = beat * 4
duration = beat * 4

# Extract the second bar (four beats) in units of seconds...
bar = amen.segment(offset=offset, duration=duration)
pq.play(bar)

# ...or, equivalently, in units of samples
m, M = int(offset * f_s), int(duration * f_s)
same_bar = amen[m:m + M]
print(f"m = {m:,}, M = {M:,}, same samples: {np.array_equal(bar, same_bar)}")

m = 76,734, M = 76,734, same samples: True

Both approaches extract the same \(M = 76{,}734\) audio samples. Use whichever units are more natural for the task at hand: seconds when working by ear, and audio samples when you need exact control.

Note that a sample extracted in either fashion does not allocate any new memory. It simply points to those audio samples in their original memory. Accordingly, any manipulations done to the sample will also apply upstream. Use pq.Audio.copy to explicitly create a copy of the memory.