Project 6 Python Option: Score Manipulation and Additive Synthesis

Project 6 Due Monday, April 7, 2025 by 11:59PM Eastern

You will submit this project via Gradescope. Report any issues on Piazza!

⚠️ Warning! ⚠️

This project requires Python knowledge and confidence with Numpy. If you find that you are struggling with Task 1, please do consider completing the Nyquist version of Project 6 instead.

Project Background: Scores and Sound Events

In this project, you will extend your previous concepts of sound design and sequencing through the usage of pyquist scores. You will re-create a popular song, through the publicly available platform TheoryTab, synthesizing a melody with a basic unit generator. You will then explore various modifications, including changing instrumentation, randomization of melody content, and more.

Important Context

A primary purpose of this alternative Project 6 is to gather feedback on the use of Python in future offerings of Intro to Computer Music. Accordingly, providing us detailed feedback in your answers.txt is as important as completing the project tasks. Please take detailed notes there while you complete the project regarding pyquist (both installation and any friction encountered while using it), the use of Python for computer music, and challenges you faced in the individual project task. Additionally, please keep track (roughly) of the time you spent on each task and include that in your answers.txt as well.

Project Setup

Install Python

Install Python 3 on your machine. Python 3.10 or greater is recommended, earlier versions of Python 3 will likely be fine as well.

Check if Python 3 is installed on your machine already by executing: python3 --version in your command line.

Recommended installation paths by operating system:

  • On Mac OS, install via Homebrew: brew install python
  • On Windows, install via official builds. Grab the Python 3.12.7 for your platform and run the installer.
  • On Linux, install via your distribution package manager. E.g., sudo apt update && sudo apt install python3 on Ubuntu.

Create and activate a virtual environment

Navigate to a directory where you would like to save your code for this course. Install virtual environment:

python3 -m pip install --upgrade pip
python3 -m pip install virtualenv
python3 -m virtualenv .venv
source .venv/bin/activate

On Windows, you will have to run .\.venv\Scripts\activate to activate your virtualenv. If python3 is not found, try python instead.

virtualenv is our official recommendation, but if you prefer, you may also use other environment tools like Conda, Docker, or even installing packages system-wide. In general, you are welcome to follow your own development norms and preferences in this course. Just be aware that the further you venture from the charted path, the less likely you are to get helpful advice from the teaching staff.

Install pyquist

pyquist is a Python library that provides some basic utilities for computer music programming. It is centered around manipulating audio data represented as NumPy arrays. The name pyquist originates through its partial inspiration from Nyquist, a computer music programming language written by Roger Dannenberg and used as a teaching language for previous iterations of this course.

Unlike other computer music programming frameworks like Nyquist, Max/MSP, Pure Data, SuperCollider, etc., pyquist does not contain a fully fledged set of “unit generators” that help with standard computer music synthesis and processing opeartions. Instead, pyquist has some simple data abstractions and helper functions that take care of the basics of adapting Python for computer music programming, and we will build up higher-level computer music abstractions throughout the course.

Follow the installation instructions to install the latest version of pyquist in your virtualenv. At a minimum, you will need to run pip install --upgrade git+https://github.com/gclef-cmu/pyquist.git (make sure you have your virtualenv activated!). This may require that you install git first.

You will likely have to run this command several times this semester as new features are added to the library. You may also consider following the official instructions to install pyquist from source—that way, updating is as simple as git pull.

Make sure you are installing pyquist on your local machine, not a remote server. The next steps will involve interacting with your local sound devices, which will not work remotely. In general, many projects in this course will require running on your local machine, so please familiarize yourself with this now.

At its core, pyquist is heavily reliant on Numpy. We strongly recommend familiarizing yourself with basic Numpy usage, such as by going through the Numpy quick start . In particular, understanding array creation, basic operations, universal functions (like sum and sin), and broadcasting rules. For instance, note that multiplying a 1 x N array with an N x 1 creates a pairwise N x N array. This can often be quite large in memory if N represents the number of samples in an audio object for instance.

Explore the pyquist CLI

Here you will familiarize yourself with interacting with Pyquist via the command line.

pyquist uses your default sound input and output devices to record and play audio. If you’d like to set a particular device instead, you can run python -m pyquist.cli devices to see and configure these. Note that by configuring your devices this way this will take precedence over your system output (e.g., if you set your pyquist output to a headphone device, and then unplug the headphones, it won’t automatically switch). To unset any device configuration, simply delete the cache file: ~/.cache/pyquist/sounddevice_defaults.json

Create a directory called project6 and navigate to this direcotry.

Next, run python -m pyquist.cli record -d 2 hello.wav to record a 2 second audio clip. Say “Hello World” while recording is happening.

Finally, run python -m pyquist.cli play hello.wav to play the audio file out of your speakers.

Within project6, create a file called hello.py with the following contents:

from pyquist import Audio
from pyquist.cli import play

play(Audio.from_file("hello.wav"))

Run python hello.py. You should hear your audio output.

FreeSound API Setup

Create and setup a FreeSound account

  • Find an initial short sound you’d like to experiment with (e.g., https://freesound.org/people/kiddpark/sounds/201159/)
  • Run the following python command to run (and optionally save) the file
  • python -m pyquist.cli freesound https://freesound.org/people/kiddpark/sounds/201159/ -o sound.mp3

Follow instructions provided on FreeSound to provide the API information:

  • Go to https://freesound.org/apiv2/apply
  • Provide some name for the API key (such as your_name_15322_cmu)
  • Provide the description in the specified box, such as “Computer music course for Carnegie Mellon University”
  • Read and accept FreeSound’s terms of use and click apply
  • Copy and paste the resulting client ID and client secret strings from the resulting page into Python
  • If you need to change this later, you can delete the api_key file in ~/.cache/pyquist/freesound/api_key.json

You can also obtain audio objects and FreeSound metadata (e.g., license info, audio description, etc.) in your own code:

from pyquist.web.freesound import fetch

audio, metadata = fetch("https://freesound.org/people/kiddpark/sounds/201159/")
  • audio is the pyquist Audio object; you can invoke functions on it (play, write, etc.), modify it, or anything as normal.
  • metadata is a dictionary containing meta information. In particular, the license key is needed whenever you utilize FreeSound in this course, as well as username for proper attribution.

Final Setup Steps

Navigate to your local project folder, project6

Additionally, install the librosa and matplotlib libraries. Note that these are only used in helper library calls; in your own code, you may only utilize pyquist, numpy, and Python built-in libraries, as per usual.

  • pip install librosa matplotlib

Task 1: Basic Score Creation and Playback

In Pyquist, a score is represented by a PlayableScore, which is a lightweight list of PlayableSoundEvent objects. A playable sound event consists of the following, as a tuple:

  • time (float), the time in seconds from the start of the score.
  • Instrument (Callable), a funciton which takes duration, pitch, and optional additional keywords and returns an Audio object.
  • kwargs (dict), the dictionary with the specified duration, pitch, and other arguments to be passed along.

Concretely, a PlayableScore can be constructed using built-in Python list, tuple, and dict types. Then, this score can be rendered into a final Audio object via the render_score function in the pyquist.score package.

Additionally, instead of using absolute time in seconds, you can use beats associated with a particular beats-per-minute (BPM) tempo. Beats in music are the regular, repeating pulses you naturally tap your foot or nod your head along to when listening to a song, with a fixed, repeating duration (e.g., 0.5 seconds at 120 BPM). BasicMetronome in pyquist.score provides a simple wrapper for converting between these two units of time. When calling render_score, a metronome object can be passed in to indicate that the time variables within each sound event are actually beats, although the “duration” parameters remain in absolute seconds.

You will begin this project by implementing hand-crafted score creation, and unit generation playback. First, fill in part1_inst according to the function signature, to be able to render a specified duration (in seconds) and pitch (in steps) with a unit sin wave.

  • Recall how Pyquist audio objects are wrappers over simple Numpy arrays, of shape n_samples x n_channels, and ensure to use a vectorized call; this function will be a building block throughout the project, so should be efficient.
  • Calling Audio.from_array on a 1d Numpy array will convert it to a mono channel audio object.

Next, implement part1_score to take in a list of note onset times, durations, and pitches, as well as an instrument to utilize, and return the created playable score.

Task 2: Envelope Generation

To obtain more interesting sound design, utilizing finer grained amplitude envelopes than the simple fade in and fade out abstractions explored last project is appealing.

A natural extension of fades is a piecewise-linear envelope. One prototypical example of this is that of the attack, decay, sustain, release (ADSR) envelope, as illustrated below [Kvifte 1989], with (ti, vi) pairs annotated for the time and value of the amplitude control points:

ADSR

Your next task is to implement a linearly-interpolating Envelope abstraction. This envelope should take two Numpy arrays as arguments, corresponding to times and values, which could be used to minimally define a piecewise curve, like above. This abstraction should also handle concretizing a curve from a list of dense times (i.e., on a sample by sample basis), via the __call__ function. If times are requested which are outside the range of the key points defined, then the nearest endpoint value should simply be extended over that portion (e.g., if the Envelope is defined from 0 to 1 seconds, and it is called with times beyond 1 second, the final defined value should be copied).

When implementing code in this task, do keep in mind Numpy rules and size warnings on broadcasting, as explained in the Project Setup.

  • A starter class has been provided in the code; fill in the sections marked with “Implement Me!”
  • Here is an example of basic expected behavior:
env = Envelope(np.array([0, 1, 2, 3]), np.array([0, 1, 0.5, 2]))
t = np.array([0, 0.5, 1.5, 2.5, 3, 3.5, 4])
print(env(t)) # should display array([0. 0.5 0.75 1.25 2. 2. 2. ])

Next, you will extend your score creation infrastructure to incorporate these envelopes.

  • Fill in part2_inst and part2_score to additionally take an Envelope object, but otherwise behave identically to their analogues in task 1. Be sure to call part1_inst and part1_score respectively to simplify your implementations.
  • If durations are shorter or longer than the defined envelope, utilize slicing and the padding described above.

Finally, you will utilize the provided get_harmonic_envelopes helper function to obtain the time-varying envelope amplitudes for multiple harmonics of a source/reference audio. This function takes a reference audio object and parameter num_harmonics to control how many different harmonics to obtain.

  • Implement part2_harmonic_inst analogously to part2_inst, but generate harmonics of the specified pitch shaped by the harmonic envelopes (obtained from get_harmonic_envelopes). Additionally account for an overall_env parameter to shape the final combined audio further, like env in part2_inst.
  • Implement part2_harmonic_score in a similar manner to part2_score, but instead utilize a provided reference audio, the get_harmonic_envelopes function, and the part2_harmonic_inst function to render multiple harmonics. Do not utilize the overall_env parameter in this case, and ensure that the number of sound events is equal to the length of the lists passed in.
  • We encourage you to explore multiple different reference audios! Note that some source audios (e.g., trumpet) sound worse than others (e.g., recorder); why is this? Which types of sounds tend to work better than others, for reproducing with sine waves and harmonics alone?

Task 3: TheoryTab Simple Cover

TheoryTab provides a community database of popular songs which have been transcribed and annotated with music theoretical information.

  • Select a song you enjoy, and then click “Open in Hookpad” and copy the URL.

In Python, you can then invoke fetch_theorytab_json(url) to download and process this information. A copy will by default be saved in your ~/.cache/pyquist/theorytab directory cache and subsequent calls will load from disk unless the ignore_cache flag is passed in. You can also manually save and load this information through the json.dump and json.load functions, ensuring to point to your relative project “data/” directory.

Next, you can convert this data to a score through the theorytab_json_to_score function, which returns a Metronome and melody and harmony Score objects. Note that elements within these score objects do not yet contain an instrument Callable. Further note that the time parameter of each sound event returned is in beats while the duration parameter within the kwargs dictionary that is passed to the instrument remains in seconds.

  • Implement part3_cover which takes in these scores, along with respective instrument parameters, and returns a combined playable score, with onset times sorted.
  • Additionally, implement a rolled duration parameter, which modifies the harmony to use rolled chords instead of blocked chords (i.e., have notes sound one after another instead of all at the same time). For sound events in the harmony which have the same original onset time (i.e., are part of the same chord), add a delay of the specified rolled duration in beats between each event from low to high pitch. With a moderate duration (e.g., a quarter of a beat), this should sound like an arpeggio.

We highly encourage you to experiment with these parameters! For instance:

  • What sort of sounds do you prefer for harmony vs. melody?
  • How does the rolled duration impact your perception of the rendered audio?
  • Are there other songs which don’t work well with the provided parameters in the starter code?

Task 4: Algorithmic Score Generation

Next, we will explore random melody and score generation. In particular, you will implement an arpeggiator which takes in a sequence of chords and their corresponding type (i.e., major or minor), along with the overarching rhythm with which to play these chords. Then, for each chord, you will randomly select a pre-defined pattern to loop over, given an initial random state. Finally, you will construct a playable score, given an Instrument to invoke.

  • Implement part4_rand in the starter code. The rhythm parameter controls how long each chord event lasts in beats, while the chords parameter contains the corresponding root pitch and chord type.
  • When selecting a new random pattern from the chord_map for each chord and its type, take care to invoke random_state.rand() precisely once per chord in the sequence to select the pattern. That is, index into the list for a given chord type like: int(random_state.rand() * len(chord_map[chord_type]))
  • Each note should last for the provided duration parameter, in beats. Use the metronome object, and the beat_to_time function, to handle this.
  • When looping over a selected pattern, jump back to the beginning upon reaching the end (i.e., like a cycle).
  • Each element of a pattern should be added to the chord’s root pitch (e.g., [0, 3, 7] on c4 would arpeggiate as [c4, eb4, g4]).

We encourage you to explore this simple arpeggiator further! What other aspects could be randomized? How can you modify the input parameters, such as the chord map or the chords themselves, to obtain interesting results?

  • Do ensure that any additions to part4_rand maintain backwards compatability with the original type signature for autograding.

Task 5: Song Cover and Composition

Now it’s time for creativity! Leverage the tools developed in this project to create a 30—60 second composition which covers (or is inspired by) one or more of your favorite songs from TheoryTab. This composition should perform a significant transformation upon the source material. Some transformations you could consider using are:

  • Combining or remixing multiple songs together.
  • Re-harmonizing a song yourself, to provide a different mood.
  • Changing the song structure, e.g., chopping or looping certain sections, etc.
  • Experimenting with different timbres of sounds, or instruments altogether, at different sections.
  • and more!

We additionally impose a few additional constraints:

  • As always, utilize envelopes so as to avoid harsh clicks or clipping.
  • For at least one instrument, you must use envelopes obtained from a reference sound, using part2_harmonic_inst.
  • You must creatively utilize part4_rand in at least one clear way.

Use the “data/” directory to save and load your json objects fetched from TheoryTab, as well as pre-computed sounds fetched from FreeSound or recorded yourself. Relative pathing should always be used within your code (i.e., json.load("data/my_song.json")).

Additionally, include a paragraph in answers.txt which details your composition intent, the modifications you made to the source audios/scores, and any other notes you would like to share.

  • For any sound obtained from FreeSound (or a similar source), include the license information in answers.txt as well. This should include a link to the license as well as any attribution if needed (e.g., for CC licenses).

Next, include a paragraph in answers.txt under “Project feedback and pain points” to share freeform feedback on the project tasking, Pyquist library usage, etc. Please provide at least one or two comments, as the course staff will greatly appreciate it and use it for next year’s version of the course!

Additionally, under “Time spent per task” please provide a rough estimate of your time spent, in hours and minutes.

Finally, run the permission_quiz function and complete your opt-in preferences for showcasing your work on the course website. This should populate the permission.json file to be include upon submission.

Submit via gradescope

Now it’s time to submit your project! Go to Gradescope and navigate to Project 6 Python. You can upload either your project6 folder directly or a zip file containing the files in the directory. Follow the feedback from the autograder until you get full marks! Note that the autograder for this project is still under development, so passing it does not guarantee your work to be completely correct; pay close attention to the requirements, and test and listen to your results with various input combinations.

Peer grading will by manually assigned and we will send out instructions after the submission deadline.

Our class late day policy will remain in place, even though any score reductions may not automatically apply on Gradescope.

References

  1. Kvifte, Tellef. Instruments and the Electronic Age: Toward a Terminology for a Unified Description of Playing Technique. Vol. 35, Ethnomusicology, 1989, doi:10.2307/852402.