Class Notes

17 March 2016

What exactly is a sample?

Digital audio is a sequence of samples. Each sample represents the amplitude of the audio signal at a point in time. "Amplitude" means pressure in air, voltage in an analog signal, displacement of a record groove -- it's all proportional and basically equivalent. We measure the amplitude 44,100 times per second to capture the essence of the continuous signal. Samples can be integers or floats. Usually hardware uses integers, but internal processing uses floats.

What is a (PortAudio) device?

A device is hardware. A device can provide audio input, play audio output, or both. In PortAudio, you open devices, so they are somewhat like files: You open and read or write data.

Blocking vs Callback API

Let's talk only about audio output. A blocking interface would be used like this:

while true:
    b = create_a_block_of_samples()
    audio_write(b)

While a callback interface would be used like this:

start_audio_stream(my_callback_function)

def my_callback_function(block)
    b = create_a_block_of_samples()
    copy b into block

With callbacks, the Audio API calls you and asks for samples. With blocking interfaces, you call the Audio API and give it samples. A callback API allows the API to provide the thread and optimize things for low latency (it might use special scheduling tricks), but then you must write multi-threaded code.

DFS?

... means "depth-first search."

Buffer Latency Math

Yes, it's confusing and intricate, which is why I didn't dwell on it. The main point is that buffers add latency, but you need buffers to provide a source of samples while you are either not running or computing new samples. The longer you might be delayed (usually because the OS is busy doing something like starting a process or reading virtual memory from the disk), the more buffers you need and the longer the overall audio latency.

Topological Sort

I searched about 50 sites to find a clear, simple presentation:
http://cs.middlesexcc.edu/~schatz/csc236/handouts/topsort.html
If you don't like it, try Wikipedia and then Google.

Think of topological sort as computing a valid ordering of courses (nodes in the graph) given a set of prerequisites (edges in the graph). Similarly, we want a valid order to compute unit generators so if A is an input to B, A is computed before B (A is a "prerequisite" to B.)

PortAudio API

Here's a tutorial: http://portaudio.com/docs/v19-doxydocs/tutorial_start.html If my summary didn't do it for you, maybe a step-by-step here's how you use it approach is better. If not, see you on Piazza?

Why Do Filters Need State?

This is a deep question, but maybe if we just look at some real filter code, it will be obvious. This is the inner loop for lowpass from Nyquist, probably based on code from csound:

Iterate the following:
    *output++ = (prev = c1 * *input++ + c2 * prev);

This filters samples in array input and write to array output. Notice that the filter modifies prev on each iteration, and the filter requires two frequency-dependent constants c1 and c2. These are all in the state for the lowpass filter unit generator.

SIMD and Vector Instructions

Intel/AMD have introduced vector instructions starting in 1997 (wow, almost 20 years!) The instructions operate on short vectors, e.g. you can compute a[i] = a[i] * b[i] or a[i] = a[i] + b[i] for vectors of length 4 to 16 (in some of the latest processors). The vectors are moved from memory to CPU registers with single instructions and the vector operations are single instructions that process the whole vector in parallel. Intel has been extending the set of vector operations over the years as new CPUs are developed. Unfortunately, compilers are not very good at using these instructions, standard C does not provide any direct way to call them, and not all "Intel" architectures implement them, so their use tends to be limited.