\textbf{[AI]}~15-622 Term Project

15-622 Term Project#

Grad students only#

Key dates#

Overview#

Every graduate student in 15-622 is required to complete an ambitious term project, instead of the more narrowly-scoped final project 15-322 students will complete.

This is your opportunity to dive deeper into an aspect of computer music that is most interesting to you, or one that intersects with your other academic, creative, or personal interests!

These projects must be completed individually (no teams). You may pursue one of three tracks.

Technical Track. The technical track involves proposing and executing a computer music software engineering project, broadly defined. Within scope would be things like:

  • Building interactive computer music applications in Python or JavaScript

  • Developing audio plugins (VST, AU, etc.) in C/C++

  • Extending open source tools we’ve used in class like Pyquist

  • Creating educational materials combining technical explanations with interactive demos

Creative Track. The creative track involves building sophisticated computer music software systems and using them to produce a satisfying creative result. While the creative result is a key goal, it is also expected that most of your creative work happen in computer music code rather than in a score editor or DAW.

  • Building a library of NumPy/Pyquist synthesis utilities and using them to realize an original piece

  • Exploring different algorithmic composition techniques and using them to create different movements of a piece

  • Recreating a famous piece of computer music in a modern computer music system

Research Track. If you prefer, you may also choose to pursue a computer music research project. However, proceed with caution, as research is harder to properly scope for a finite timeline. If you pursue this route, expect to work w/ the instructor to ensure that your project has a high likelihood of being achievable before the end of the semester. Within scope would be things like:

  • Implementing classical techniques from existing computer music research papers in Python or other languages

  • Deploying a modern music AI technique on synthetic data and measuring generalization to real-world data

  • Evaluating the computer music capabilities of commercial AI systems like ChatGPT or Claude

  • Training existing methods on new datasets and measuring performance

More project ideas#

These were implemented by students or suggested in prior years of the course:

  • VSCode extension for computer music development

  • Creating a YouTube video explaining algorithmic composition using manim

  • Web-based compressor in the Web Audio API

  • Real-time C++ vocoder in JUCE

  • Real-time distortion plugin in Faust

  • Interactive guitar tuner application using Pyin

  • Toolkit for creating synthwave music

  • Interactive web-based music sequencer

  • A plugin for integrating Vocaloid into a game development engine

  • Original composition built on a sophisticated library of computer music synthesis and processing primitives

  • System to sonify chess replay data

  • Reverse engineering the time-varying FM synthesis parameters in John Chowning’s “Stria”

  • Creating renditions of classical music (a la Wendy Carlos or Isao Tomita) using handcrafted synthesis algorithms

  • An infinite MIDI generation tool (demo)

  • Modeling specific composers using Markov chains and using them to quantify cross-composer similarity

  • Using commercial AI systems to convert text descriptions into plugin parameters

  • Evaluating commercial AI systems on writing code in domain-specific computer music languages

AI Policy#

AI usage is acceptable / encouraged for this project. However, the standard AI policy applies. Particular guidance for this project includes:

  • You are expected to retain most of the agency when developing the code for your project. You tell the AI what to do in an iterative, granular fashion, rather than asking it to complete large swathes of your project for you automatically.

  • Writing = thinking. Try to do most of the writing yourself for the reports. You may use AI to help, e.g., to convert a detailed outline into prose.

  • Take ownership of everything you submit. You must review all AI contributions to your project and be able to answer technical questions about your submitted work.

  • You are not allowed to use any AI music or image generation (e.g. Suno), unless discussed with and approved by the instructor in advance.

  • Failure to comply with the AI policy may result in significant penalties to your project grade. We may conduct a Debugging and Reasoning Check if we suspect that you used AI inappropriately to complete your project.

  • High-level: the main thing we are looking for here is evidence of significant human effort (on the order of 20-40 hours of work over the course of the semester). As long as you are clearly putting in effort into this project rather than just using AI to complete the whole project the night before the deadlines, you should be fine.

Milestones#

The project has several milestones over the remainder of the semester:

Milestone 1: Proposal#

Due Monday, October 5 by 11:59PM. May be submitted up through Friday, October 9 if you need additional time.

A written proposal is required. Your proposal should be about 2-3 pages and contain the following sections.

  • Overview. The general nature of what you will build, described at a high-level.

  • Steps. The individual steps you will take to achieve your goal. Be as specific as possible, e.g., “Implement an AudioNode in JavaScript for real-time FM synthesis with time-varying control.”. Include at least 5 steps, in the rough order in which you will complete them. Make sure to include any code, journal articles, etc. you will rely on, port, implement, modify, etc.

  • Requirements. Describe the measurable requirements of the final deliverables you plan to submit. Be as specific as possible, e.g., “Library contains five synthesizer patches exploring different time-varying FM synthesis parameters”. Include at least 5 requirements. It’s okay for the requirements and steps to be somewhat redundant.

  • Interim Milestone. Which of the numbered steps / requirements will you have completed by the interim milestone on November 9th (think about what you can realistically accomplish in about a month!)

  • AI Usage. Explain as specifically as possible how you plan to use AI in the completion of your project (or remark that you don’t plan to use AI at all).

See the steps / requirements lists in the three Assignment 3 directions to get a sense of the specificity we’re looking for.

All projects must be approved by the instructors, so this really is a proposal. We will usually suggest some changes, and if the project really seems out of line, we’ll let you know early and without penalty. As long as your proposal is clear, even if the project is infeasible, you won’t be penalized for your proposal.

SUBMISSION: Submit your proposal (as a PDF file) by email to icm-csd@cmu.edu with the subject “15-622 Final Project”.

Milestone 2: Progress Report#

Due Monday, November 9 by 11:59PM.

A progress report is required. Your report should be about 1-2 pages and contain the following sections:

  • Progress Reflection. A general self reflection on the progress you’ve made so far. Remark on whether you feel on track, ahead of, or behind schedule.

  • Steps / Requirements Completed. Review the steps / requirements you submitted in your proposal. Which of them have you completed? Does this align with what you planned to have accomplished by the interim milestone?

  • Steps / Requirements Adjustments. Remark on any adjustments you are making to the steps / requirements based on your progress so far.

SUBMISSION: Submit your report (as a PDF file) by replying to the same email thread you used to submit your proposal (subject “15-622 Final Project”).

Milestone 3: In-class presentation#

You will present during the last week of class on either Tuesday, December 1 or Thursday, December 3 (you will be assigned to one or the other).

Use slides to supplement your talk. Plan to present for 7 minutes maximum. This is a very short presentation.

Please rehearse your talk. It is expected to be polished, precisely timed, and effectively communicate your achievements w/ the undergraduate students.

Due to the tight time requirements, you will not be able to directly plug your computer into the projector. There are two presentation options:

  1. (Recommended) Upload your slides to a shared Google Drive folder (link TBD). Google Slides, PowerPoint, Keynote are all fine. Make sure all audio is embedded. For Google Drive, make sure I have permission to view your embedded audio/video.

  2. Screen share from your own device via Zoom. This is preferable if you have an interactive demo to share. You should anticipate some latency from the Zoom setup. Contact the instructor to test ahead of time!!

A suggested presentation format:

  1. Motivate your work. What purpose (outside of your education) is served by your project? What existing problem does your work solve? (1 slide)

  2. State your approach and goals. How does your project solve the problem just stated? If it’s an interactive application or involves sound, please give a demo. Be sure to state clearly and with some detail exactly what you did. (1 or 2 slides)

  3. What is the state of your project? Tell what is complete and working as well as what would be the next step if you (or someone else) were to continue working. (1 slide)

If you plan to show sound examples or demos (recommended):

  • be prepared to play sound files from your computer or embed them w/ proper permissions into your presentation

  • be sure you know how to share audio in Zoom and test this with a friend to make sure the sound levels are reasonable

  • be ready to show any interactive demo you may have, though also have a pre-recorded demo video as a backup!

Milestone 4: Final Deliverables and Report#

Due Friday, December 4 by 11:59PM.

You are required to submit both your final deliverables (code, video, audio, etc.) alongside a final report.

Your final deliverables should be a complete package including but not limited to code, README documentation, and examples. This should contain everything we need to execute your project. For example, if your project were to create a phase vocoder, you would submit the software implementation, a README documenting how to apply the vocoder and use all the parameters, provide code examples that apply the vocoder to a test sound, and one or more sounds that illustrate the effect of the vocoder.

Your final report should be 3+ pages and contain at least the following sections:

  • Overview. Describe your completed project and what you have accomplished. This should be standalone (don’t assume the reader has read your proposal or progress report).

  • Technical Approach. Describe the key technical decisions and implementation details of your project. What languages, libraries, or frameworks did you use? What was the high-level architecture or design? Include enough detail that a technically-proficient reader could understand how your system works. A picture is worth a thousand words - figures are strongly encouraged!

  • Results and Demonstration. Showcase the results of your project. Reference specific audio examples, screenshots, videos, or other outputs included in your deliverables. For creative track projects, discuss how your technical work serves your creative vision and how the final piece turned out. For research track projects, present your findings, analyses, plots, results tables, etc.

  • Challenges and Lessons Learned. What were the most significant challenges you encountered? How did you address them? What would you do differently if you were starting over?

  • Steps / Requirements Completed. Restate the steps and requirements you submitted in your proposal and progress report. Which of them have you completed? Which, if any, did you fall short on? Be honest here — we are more likely to give partial credit for honest shortcomings than for incomplete work framed as completed.

  • AI Usage. Explain as specifically as possible how you used AI in the completion of your project (or remark that you didn’t use AI at all).

SUBMISSION: Compress the final deliverables in a zip file and final report as a PDF, submit by replying to the same email thread you used to submit your proposal (subject “15-622 Final Project”).

Evaluation#

  • (15%) Proposal submitted as instructed

  • (15%) Interim report submitted as instructed

  • (20%) Effective in-class presentation

  • (50%) Final deliverables meet stated requirements, final report submitted as instructed