10.5 Real-time processing

Frame-based processing has one more role to play, which we will return to in Chapter 17: it is how real-time audio works. Suppose we want to synthesize an endless stream of audio on the fly, computing each sample \(x[n] = x(\tfrac{n}{f_s})\) just in time to be played. We could call our synthesis function once per sample, but function calls have computational overhead, and at tens of thousands of samples per second that overhead adds up fast. Worse, it is overkill: we cannot physically turn knobs fast enough to need per-sample control anyway.

Instead, real-time systems compute audio in frames, usually called blocks in this context. We pick a block length \(B\), and at each moment \(\frac{k \cdot B}{f_s}\) the operating system asks our program for the next \(B\) samples. This is exactly frame-based processing with \(N_H = N_F = B\). As long as we can compute each block in less than \(\frac{B}{f_s}\) seconds, the audio never runs dry and we achieve a real-time stream. We will develop this idea properly when we study real-time, interactive audio.