Direction 2 (Technical): Clone an acoustic instrument using additive synthesis

Direction 2 (Technical): Clone an acoustic instrument using additive synthesis#

With your skills in additive synthesis, you’re already able to create simple but convincing sounds that emulate a real instrument. In this direction, you will pick a recording of a real acoustic instrument, analyze what makes it sound the way it does, and rebuild it from nothing but sine waves.

To clone a pitched instrument using only additive synthesis, you need to recreate three behaviors: its harmonic spectrum (which harmonics are present, and how loud each one is relative to the fundamental), its amplitude envelope (how the overall loudness evolves from attack through release), and its fundamental frequency (which must follow whatever pitch you ask it to play). The steps below walk you through analyzing all three — recreating them is up to you.

Note

Some instruments use vibrato, a slight oscillation in pitch. You cannot achieve this with additive synthesis alone, but you’ll have the opportunity to implement it in Assignment 4.

Steps#

Download the starter code file 3-2-clone.ipynb. Unlike for the autograded assignments, this starter code is merely a suggestion. You’re free to implement however you want, assuming it conforms to the requirements below.

  1. Find a sound on FreeSound or Sample Focus. You want one clean tone of your desired instrument: your instrument playing a single note at a time, uninterrupted, with no background noise and no reverb tail from another instrument. These aren’t hard to find, but don’t be afraid to go to “page 3” to find one — a clean sound makes every following step dramatically easier. Load it into Pyquist with pq.Audio.from_file(fp), or fetch it directly with fetch_freesound, which also returns the metadata you need for attribution.

  2. Analyze the amplitude envelope. Use pq.plot(audio) to visualize the waveform, and identify the attack, sustain, and release phases of the sound. This is more of an approximation than the spectrum analysis, but you should be able to get a good idea of how long each phase lasts and how loud the sustain is relative to the peak.

  3. Analyze the harmonic spectrum. Use pq.plot_freq(audio, offset=..., duration=...) to visualize the magnitude spectrum of a short window of the sound. Identify the fundamental frequency, then the amplitude of each harmonic relative to it. Be deliberate about which window you analyze: the spectrum during the attack transient looks quite different from the spectrum during the sustain. pq.plot_spec(audio) will show you how the spectrum evolves across the whole note.

  4. Implement your cloned instrument. Using what you measured, implement cloned_instrument(pitch, duration, sample_rate=44100, **kwargs) -> pq.Audio as a sum of sinusoids shaped by an amplitude envelope. It must be able to play a note at any pitch and any duration and still sound like your instrument.

  5. A/B your clone against the original. Render a single note at the same pitch and duration as your source recording and save it as clone.wav. Listening to source.wav and clone.wav back to back is the fastest way to hear what your analysis missed; iterate on your harmonic amplitudes and envelope until you’re happy with the match.

  6. Hear your instrument play a song. The starter code provides play_song(), which fetches a TheoryTab melody, renders it with your cloned_instrument, and writes out song.wav. This is where you’ll find out whether your instrument holds up across the whole pitch range, rather than just at the one pitch you analyzed.

Requirements#

  • Your source sound is a recording of a real, pitched, acoustic instrument playing a single clean note. Percussion, unpitched sounds, chords, and synthesizer presets do not qualify.

  • Your instrument is implemented as a function with the signature cloned_instrument(pitch, duration, sample_rate=44100, **kwargs) -> pq.Audio, where pitchis a MIDI pitch number andduration is in seconds.

  • The returned audio is duration seconds long at the requested sample_rate.

  • pitch controls the fundamental. Calling your instrument at two different pitches must produce two sounds with correspondingly different fundamental frequencies — the pitch cannot be hard-coded to your source recording’s.

  • Your synthesis is purely additive: the output is a sum of sinusoids that you generate. Playing back, resampling, pitch-shifting, or otherwise transforming the source recording itself does not count.

  • Your instrument sums at least 4 harmonics, with relative amplitudes that you individually chose based on your spectrum analysis (i.e., not all equal, and not an off-the-shelf formula).

  • Your instrument applies an amplitude envelope with distinct periods, based on the envelope you measured.

  • clone.wav is a single note rendered by your cloned_instrument at the same pitch and duration as source.wav, so that the two files can be compared directly.

  • song.wav is produced by rendering a score with your cloned_instrument, and contains at least 8 different notes.

  • TECHNICAL.md reports the actual numbers you measured (fundamental frequency in Hz, relative harmonic amplitudes, envelope phase durations in seconds), and those numbers match what your code uses.

Submission#

Your submission must conform to the technical format. Bundle the following into a single .zip file (under 100MB) and submit it via Gradescope.

  1. demo.mp4, demo.mov, or demo.mkv — a 120-300 second screen recording, narrated in your own voice, explaining your approach and demonstrating your result. See the technical format for the required presentation flow.

  2. TECHNICAL.md — use the open-ended template and leave all ## Section Headings unchanged. Your write-up must state:

    • The type of instrument you cloned.

    • The source of your recording and any required attribution (author, link, license).

    • The fundamental frequency you measured, in Hz.

    • The relative amplitudes of the harmonics you measured, and how many you synthesize.

    • The attack, sustain, and release of the envelope you measured, with durations in seconds.

    • How you turned all of the above into your cloned_instrument implementation.

  3. src/ — your Python/Pyquist code plus any sound assets it needs. Anything we can pip install does not need to be included. At minimum this directory should contain something like:

    • clone.ipynb or clone.py — your implementation of cloned_instrument.

    • source.wav — the original reference recording you analyzed.

    • clone.wav — a single note from your cloned instrument, at the same pitch and duration as source.wav.

    • song.wav — the song rendered with your cloned instrument.

Note

The Gradescope autograder only checks the top-level structure of your zip file. It does not verify the specific contents of your src/ directory. Making sure the files above are present, complete, and runnable is up to you.