Direction 3 (Technical): Implement tap detection to create scores

What if you could create the rhythm of a score just by tapping your finger? In this direction you will build a tap detector: audio in, a list of tap times out, and a score rendered from those times.

Tap detection sounds trivial and isn’t. A tap is not an instant — it’s a burst smeared over tens of milliseconds, so even the “true” moment is a judgment call. The waveform oscillates, so a single tap crosses any fixed amplitude threshold many times. Taps vary in loudness, surfaces ring and bounce, and rooms have noise floors. There is no obviously correct algorithm here, which is why the point of this direction is not just to write one, but to measure it.

Steps

Download the starter code file 3-3-taps.ipynb. Unlike for the autograded assignments, this starter code is merely a suggestion. You’re free to implement however you want, assuming it conforms to the requirements below.

1. Record. Capture at least 5 seconds of tapping with pq.record and save it as tapping1.wav.

2. Label. By hand, write down the time of every tap in that recording, to roughly millisecond resolution — e.g. [0.312, 0.804, 1.247, ...]. Use pq.plot, zooming in with offset and duration, to read the times off the waveform. This is your ground truth.

3. Detect. Implement detect_taps, returning a list of tap times in seconds. The algorithm and its parameters are entirely up to you; time-domain and frequency-domain approaches can both be made to work.

4. Evaluate. Score your detector against your labels with the provided evaluate_taps, which wraps mir_eval.onset.f_measure. A prediction counts as correct if it lands within 50ms of a label. Iterate on your algorithm and its parameters until precision and recall are both 1.0.

5. Sonify. Turn the detected taps into a pq.Score, render it with an instrument of your choosing, and save the result as song.wav. Your taps give you rhythm but no pitch, so you have to invent one. A few directions:

   • Cycle a fixed melody, scale, or arpeggio, advancing one step per tap.

   • Randomize within a scale, so the rhythm is yours but the tune isn’t.

   • Follow a progression, moving to the next chord every few taps and drawing pitches from it.

   • Derive pitch from the tap itself — map each tap’s loudness to a pitch, so how hard you hit picks the note.

   • Derive pitch from the rhythm — map the gap since the previous tap to a pitch, so fast runs and slow ones land in different registers.

6. Generalize. Record and label a second take, tapping2.wav. Run your detector on it with exactly the settings you tuned on the first take and report precision and recall. You are not required to fix any drop you see — but you are required to report it and say what you think caused it.

Requirements

• tapping1.wav and tapping2.wav are your own recordings, each at least 5 seconds long and containing at least 10 taps.

• Both recordings are hand-labeled, and both label lists appear in your submitted code.

• You implement detect_taps(audio: pq.Audio, ...) -> list[float], returning tap times in seconds. Any parameters you like; whatever you add must have defaults so the function can be called the same way on both takes.

• You implement the detection yourself. Calling an off-the-shelf onset detector (e.g. librosa.onset.onset_detect) does not count. mir_eval is for scoring only.

• On tapping1.wav, your detector achieves precision = 1.0 and recall = 1.0 at the 50ms tolerance.

• On tapping2.wav, your detector runs with identical settings — no re-tuning — and you report the resulting precision and recall, whatever they are.

• Your score has one event per detected tap, each event’s time is that tap’s time in seconds, and each event’s kwargs include "pitch".

• song.wav is that score rendered via pq.Score.render, and its rhythm audibly matches tapping1.wav.

Submission

Your submission must conform to the technical format. Bundle the following into a single .zip file (under 100MB) and submit it via Gradescope.

1. demo.mp4, demo.mov, or demo.mkv — a 120-300 second screen recording, narrated in your own voice, explaining your approach and demonstrating your result. See the technical format for the required presentation flow.

2. TECHNICAL.md — use the open-ended template and leave all ## Section Headings unchanged. Your write-up must state:

   • How you labeled your recordings, and how confident you are in the labels.

   • How your detector works, and what its parameters ended up as.

   • Precision and recall on both takes, reported as numbers.

   • What the difference between the two tells you: what did your detector learn about take 1 specifically, rather than about taps in general?

   • How you determined the pitches for your score.

3. src/ — your Python/Pyquist code plus any sound assets it needs. Anything we can pip install does not need to be included. At minimum this directory should contain something like:

   • tap_detection.ipynb or tap_detection.py — your implementation, including both label lists.

   • tapping1.wav and tapping2.wav — your two tap recordings.

   • song.wav — your rendered score.

Note

The Gradescope autograder only checks the top-level structure of your zip file. It does not verify the specific contents of your src/ directory. Making sure the files above are present, complete, and runnable is up to you.