10.6 Summary

10.6 Summary#

  • Frame-based processing operates on music at intermediate, musically intuitive rates by aggregating samples into frames.

  • A frame is \(x_k[n] = x[k N_H + n]\), controlled by the frame length \(N_F\) and the hop length \(N_H\). Their ratio sets the overlap.

  • Overlap-add reassembles frames. It is perfect reconstruction when \(N_H = N_F\) with rectangular windows, or more generally when windows satisfy constant overlap-add.

  • Granular synthesis chops sound into grains and manipulates them (reordering, resizing, respacing) to create textures, and to time-stretch without changing pitch.

  • The short-time Fourier transform takes the DFT of each frame, revealing how frequency content evolves over time. Its magnitude is a spectrogram.

  • The frame length \(N_F\) trades time resolution against frequency resolution; the hop length \(N_H\) trades time resolution against computation. Windowing each frame reduces spectral leakage.

  • The STFT is invertible, enabling spectral processing: editing a sound in the time-frequency domain. The phase vocoder uses this to time-stretch while preserving pitch.

  • Real-time audio is frame-based processing with \(N_H = N_F = B\), computing one block at a time.