\textbf{[AI]}~15-322 (Undergrad) Assignment 9

15-322 (Undergrad) Assignment 9#

Creative Tool#

Important

This assignment is for 15-322 (undergraduate) students only. 15-622 students do not complete this assignment; see the 15-622 (Grad) Grad Composition Project instead.

This is an open-ended assignment in the technical format. Carefully review the open-ended submission instructions and policies before you begin.

Due: Mon Nov 16 2026 by 11:59PM Eastern#

Overview#

In this assignment, you will take one of the advanced techniques from the second half of the course (frame-based processing onwards) and use it to build a creative tool: a piece of software that you will use to make or shape sound.

The twist: you must use your tool yourself in your Final Composition Project, a computer music composition that will be played in class in Week 14. Build something that you actually want to make music with.

“Creative tool” is defined broadly. Whatever form it takes, your tool must:

  1. Be built around an advanced technique that you implemented yourself (see Choose a technique).

  2. Synthesize or transform audio, because you will use it to synthesize or modify sound in your final composition.

  3. Be controllable. It exposes parameters (function arguments, command-line flags, GUI sliders, MIDI CCs, keys, …) that audibly change the result.

Beyond that, the form your creative tool takes is up to you. All of these are fair game:

  • A library of synthesis and/or effect functions that you call from your composition code (e.g., a Python module of granular synthesis and spectral effects)

  • A synthesizer or sound generator with a set of controllable parameters

  • A generative system that produces scores or sound from a few high-level controls

  • A command-line program that processes audio files

  • A real-time instrument you play with your computer keyboard, a MIDI controller, or your microphone

  • An interactive web app built on the Web Audio API

  • A VST/AU plugin (e.g., in C++ with JUCE)

  • A practice tool for an acoustic instrument (e.g., a harmonizer or looper that responds to what you play)

We strongly recommend that you use Python / NumPy / Pyquist to implement your tool, but you may also implement your tool in other programming languages or frameworks, if you prefer.

Choose a technique#

Your tool must be built around at least one advanced technique from Chapters 10–16 (frame-based processing onwards). The table below lists suggestions only: you may use any technique from these chapters, and you are encouraged to invent your own tool ideas!

Chapter

Technique

Example tool ideas

Ch. 10

Frame-based processing and the STFT

Time-stretcher / pitch-shifter, spectral freeze, spectral gate, cross-synthesis (“make my voice sound like a cello”)

Ch. 10

Real-time processing

Any of the ideas on this page, but playable live with low latency

Ch. 11

Sample-based synthesis

Granular synthesizer, multi-sample instrument built from your own recordings, beat slicer / resequencer, concatenative synthesis from a sound corpus

Ch. 12

Physical modeling

Karplus-Strong electric guitar, waveguide flute, modal synthesis of struck objects, “impossible instrument” models

Ch. 13

Delay lines, effects, and spatialization

Algorithmic reverb, chorus / flanger / phaser, multi-tap rhythmic delay, binaural panning

Ch. 14

Algorithmic composition, Markov chains, LPC

Markov melody generator trained on MIDI files, generative rhythm engine, LPC talkbox / vocoder

Ch. 15

Psychoacoustics, compression, MIDI

Frequency-domain compression as an effect, Shepard-tone generator, MIDI arpeggiator or harmonizer

Ch. 16

Music information retrieval and Music AI

Pitch-tracking harmonizer, onset-triggered effects, beat-synced slicer, a small model you train yourself

Note

Chapters 15 and 16 are covered in lecture after this assignment is due, but you are welcome to read ahead and build on them.

Note

You may also build on a technique from Chapters 1–9 (e.g., wavetables, FM, filters, convolution), as long as your tool clearly goes well beyond the scope in which the technique was studied in the book and assignments. For example, a wavetable synthesizer that extracts and morphs between wavetables taken from your own recordings would qualify. Re-packaging your Assignment 5 DX7 would not. If in doubt, ask early on Piazza.

Scope#

You will be graded on technical achievement and how well your tool works, not on how polished it looks. The Scope row of the requirements is graded against these examples:

Example

Too small

Your Assignment 7 granular time stretcher, unchanged, wrapped in a function with a stretch_factor argument.

On target

A granular synthesizer that takes any sound file and exposes grain size, density, pitch spread, and other extraction/reassembly behaviors, demonstrated on three very different source sounds.

Ambitious

A real-time spectral freeze instrument you play from a MIDI keyboard, where each key freezes the live microphone spectrum transposed to that pitch, with controls for blur and decay.

Starting from your own code from earlier assignments (e.g., Assignments 7 and 8) or from textbook examples is encouraged. You will be assessed on what you add.

Implementation rules#

The core of your technique must be implemented in your own low-level code. You may use high-level libraries for support around the technique, but not for the technique itself.

  • OK: NumPy / SciPy primitives (FFT, basic filtering), Pyquist, audio and MIDI I/O (sounddevice, mido, JUCE’s audio/MIDI plumbing, Web Audio API nodes for I/O), GUI frameworks, file loading.

  • Not OK as your core technique: a single library call that is the technique, e.g., librosa.effects.time_stretch, pedalboard.Reverb, Rubber Band, a Web Audio ConvolverNode as your reverb, or a commercial AI music service (e.g., Suno).

If you’re unsure whether a library is OK, ask on Piazza.

Requirements#

Each requirement is worth the points listed. These 80 points are the manually graded part of your grade. The other 20 come from the Gradescope formatting check.

Requirement

Points

R1

Technique. Your tool is built around at least one advanced technique from Chapters 10–16, or a substantial extension of a technique from Chapters 1–9 (see Choose a technique). You name the technique(s) in TECHNICAL.md, and the technique is central to what the tool does, not a minor add-on.

10

R2

Implementation. The core of the technique is implemented in your own code (see the implementation rules), and the tool works as you describe it.

15

R3

Controllable. Your tool exposes at least 3 user-adjustable parameters that meaningfully and audibly change the result, so that it can produce diverse results rather than one fixed sound. TECHNICAL.md documents each one (name, range, and what it does).

20

R4

Demo. Your video demonstrates at least 3 different examples (different inputs and/or settings, as diverse as you can make them) and explains how your technique works, pointing to the relevant parts of your code.

15

R5

Composition plan. TECHNICAL.md describes how you plan to use your tool in your final composition, and your video plays a short sketch (at least 10 seconds) of material you made with the tool that you might use in your composition.

10

Scope. Graded against the scope examples. 10: on target or beyond. 5: somewhere between too small and on target. 0: too small. Ambitious tools may also earn bonus points (see Grading).

10

Total

80

Demo video#

Your demo video must follow the technical format (2–5 minutes, narrated in your own voice). For this assignment, use this presentation flow:

  1. (15–30s) What your tool is, what technique it is built on, and what it’s for.

  2. (60–120s) Demo at least 3 different examples, as diverse as you can make them. Show the controls changing the sound.

  3. (60–90s) How the technique works in your code. Show the relevant code.

  4. (15–30s) Play your composition sketch and say how you plan to use the tool in your final piece.

If your tool runs in real time, show it running live (a screen recording with system audio, or a phone video of you playing it, is fine).

Submission#

Your submission must conform to the technical format. Bundle the following into a single .zip file (under 100MB) and submit it via Gradescope.

  1. demo.mp4, demo.mov, or demo.mkv: your 2–5 minute demo video.

  2. TECHNICAL.md: use the open-ended template and leave all ## Section Headings unchanged.

    • Under ## Requirements, write one numbered item per requirement (R1–R5) explaining how you met it. For R3, list each parameter with its range and effect.

    • Under ## Code execution instructions, explain how to run your tool, including any build steps, dependencies, and hardware or OS requirements (e.g., “needs a MIDI keyboard”, “macOS only”).

  3. src/: the source code for your tool, in any language, plus any assets it needs (sound files, presets, etc.). Leave out large frameworks and build artifacts (e.g., the JUCE library, node_modules/, compiled binaries). Anything we can pip install or download does not need to be included.

Tip

You may keep improving your tool after you submit. Your Final Composition Project only requires that you use a version of your Assignment 9 tool. Bug fixes, new features, and new controls are all welcome. Note any changes in your final project write-up.

Grading#

Grading follows the open-ended grading policy: the 80 points above plus 20 points for passing the Gradescope formatting check. As with other open-ended assignments, up to 10 bonus points may be awarded for going above and beyond, e.g., a tool in the ambitious range of the scope examples.

The standard late policy applies, but remember that your Final Composition Project is due two weeks later and depends on this tool.