Projects

Every student is required to complete an extended project in the second half of the semester. Different phases of this are Projects 7, 9, 10, 11, and 12. The project combines the development of some software with the creation of music using that software. Students may form teams, but it is essential that team members each take responsibility and credit for some aspect of the project. Team members may chose to create separate compositions and only collaborate on software. Alternatively, a "composition team" may form from different technical projects to combine multiple techniques in a single composition.

An Example Project

An example is the best way I can describe the features of a good project, so here it is: The latest Nyquist IDE (based on a project from a previous class) is great for editing text and displaying sounds (at least short ones), but there is no support for entering functions of time. This project adds a new view to the IDE allowing the user to edit piece-wise linear functions of time. Each function has a name, and multiple functions can be stored in one file. The functions are implemented by defining a Lisp function containing a call to PWL. If the user creates a time function named foo in the IDE, then the user can write (foo) in Nyquist code to obtain the function. When the user opens one of these files in the IDE, the IDE parses the code and can then display functions graphically. The user can add, move, and delete breakpoints, change function names, copy functions to a new name, etc. It would be great to be able to display multiple functions in different colors so they can be edited in concert. Also, it should be possible to snap to user-controlled grid lines, zoom in or out, change the vertical axis, etc.

Having convenient direct-manipulation access to functions of time opens up a new way to control synthesis. Complex sounds with several parameters can be made to evolve according to time-varying controls that are edited graphically. These parameters in the editor become a sort of musical score notation. The composition associated with this project uses this approach to composition, resulting in music with slowly-varying, but very carefully controlled and refined textures.

Why is this good? It has the following features that I am looking for:

	It's Computer Music: there's computation, and there's music.
	It requires some thought about music representation and music processing.
	It extends Nyquist (or Audacity). Notice that the project is not even a complete stand-alone system, only the extension of one, so the student does less work and leverages what's already there.
	The project isn't too easy, but it's not impossibly hard.
	Students will be able to use this in the future. (In fact, most of this has been done -- try it!)
	The music is integral to the project. It's almost obvious what the music has to be and how the implementation is essential to the music creation. The music and the technology guide each other.
	The music is not conventional, popular music that might be better played on a piano. Instead, the music could only be created by computer.

Your project will probably not have to have all the features, but I hope this give you a good idea of what I am looking for. Review the list above with respect to your project.

The Extended Project and its Parts

The project has several parts:

Project 7: Proposal (due March 4)

A proposal is required. Your proposal should state clearly:

	the general nature of what you will implement,
	what code, journal articles, etc. you will rely on, port, implement, modify, etc.,
	a significant milestone that you will report/deliver as the interim report
	a specific list of deliverables for your final project submission
	how will you measure your success? E.g. what functionality will you demonstrate?

All projects must be approved by the instructor, so this really is a proposal. I will usually suggest some changes, and if the project really seems out of line, I'll let you know early and without penalty. (A clear proposal for a bad project gets a good grade and a request for another submission, a vague proposal for a good project gets a bad grade and an OK to proceed).

Copy your proposal (as a PDF) to your afs project folder proj-07 using the filename andrewid.pdf, as usual. The deadline is 10:00pm, Wednesday, Mar 4th.

Project 9: Interim Report (due Wednesday, April 1)

An interim report is required. Ideally, you will have completed the milestone described in your project proposal. If you fail, you should describe clearly what you have done, what problems you encountered, whether this will affect your ability to finish the project as planned, and any modifications (subject to instructor approval) you feel you should make to the project. You should submit interim code, sound results, and other data to substantiate your report. Put everything in proj-09 in your afs hand-in folder.

Project 10: The Composition (due Wednesday, April 22)

Ideally, your composition project will demonstrate what you have learned in the class. Aim for a length of about 5 minutes. Longer works are acceptable, but length will be considered when we program the final concert. You should have some interesting ideas from the in-class listening sessions, and your project implementation will also guide you. It is probably better to make a piece in the tradition of contemporary art music than to create a rap tune, but the instructor will do his best to stay open-minded. Regardless of the musical genre, the music should reflect careful thought and imagination. On the technical side, the music should show some mastery of the techniques taught in class. E.g. a piece that is acoustically dry, monophonic, exhibits clipping, shows no variation in timbre or articulation, and which uses a completely flat tempo will not be acceptable. A piece should include appropriate reverberation, stereo placement, effective changes in dynamics (without clipping), timbres that evolve in interesting ways. and sounds shaped into flowing phrases. These are not easy requirements -- it takes time and effort to produce musical results from computers. Grading will be based on how well the piece communicates a musical idea, i.e. the intentions of the composer, and the extent to which the piece was carefully constructed using techniques learned in class and developed in the project. To a large extent, the grade will be based on effort. It is usually clear whether a piece was thrown together quickly or whether many details were carefully adjusted. If you feel you have worked very hard and still are not satisfied with the result or think that your effort is not apparent in the composition, you should submit early drafts and explain what you have done to refine the work.

What to turn in:

Copy the final piece to your proj-10 folder in your afs area. The preferred format is 16-bit, stereo, PCM at 44100. If you have more or less than 2 channels, you might want to make a stereo mix for inclusion on a CD we plan to create for the class. You can then submit both your concert version and the stereo mix.
Program notes: a short paragraph about your piece, suitable for inclusion in a concert program.
Short bio: a short paragraph about you, suitable for inclusion in a concert program.
Code: source code you used to create your piece. We realize you may have used other non-programming techniques but would like to see whatever code you wrote.
Technical notes: anything else you want to tell us about your piece and how it was created (not for inclusion in program notes).

Example program notes and bio:

The Journey of the Fly
by Charles-Christopher Onyeama

My piece was designed to outline the journey of one fly, from getting lost from his home, to finding a new resting place where he could be happy. It involves heavy sampling of various noises from everyday life, put together in what I think is a very nice tale. I used samples that I found from all over the Internet, combined with techniques such as panning, envelopes, granular synthesis, sound modulation, etc. This involved heavy use of Audacity.
I am an ECE major, minoring in CS. I consider myself an analytical/inquisitive person about people and the world around me, and as such, I like listening to modern electro-acoustic music to try to examine what the composer was trying to convey. My interests include manga, video games, etc., and I am from Los Angeles, CA.

Project 11: The Class Presentation
Talks: Thursday, May 7, 8:30-11:30 in the Exam period
Slides: due Wednesday, May 6th, by noon

Yes, I know the project number is out of order. This year, we have moved the presentations to the final exam period due to the large number of projects.

You will present your project to the class. Plan for 4 minutes maximum. This is a very short presentation. You should present your work in three parts. Use slides (preferably PowerPoint) to supplement your talk.

Motivate your work. What purpose (outside of your education) is served by your project? What existing problem does your work solve? (1 slide)
State your approach and goals. How does your project solve the problem just stated? Be sure to state clearly and with some detail exactly what you did. (1 or 2 slides)
What is the state of your project? Tell what is complete and working as well as what would be the next step if you (or someone else) were to continue working. (1 slide)

Team projects will get 1 extra minute per person (3 + n minutes for n people), and each person in the team should talk about their individual contributions or the aspect of the project they know the best.

You must submit PowerPoint or PDF slides in advance to your hand-in direcotry: proj-11 in your afs hand-in folder.

These will be organized into one monster file so that we have zero setup time between class presentations.

If you have sound file examples (recommended):

	collect them all in a directory named your-andrew-id
	at the appropriate points on your slides, put something like "play: filename"
	submit a zip file of the whole directory to proj-11 in your afs hand-in folder.
	we will unzip the file and it will be available for your class presentation
	do not assume that links from PowerPoint to soundfiles will be maintained when your slides are merged into one big file.

Assuming a 4-minute talk, you can rehearse it 5 times in 20 minutes. Do it! (If this seems unnecessary and you are not a musician, do it just to learn what musicians know that you don't -- seriously.)

The Concert (May 1, Wean Hall 7500, 6PM sound check, 7:00PM concert )

Your friends are invited. Attendance is required. This is a large class, so it will not be possible to perform every piece. The instructor and teaching assistants will select pieces if necessary to keep the program length reasonable.

Project 12: The Final Project Submission (due Friday, May 1)

Your goal is to make a complete package of software, documentation, example code, and sound examples. For example, if your project is to create a phase vocoder, you should submit the software implementation, document how to apply the vocoder and use all the parameters, provide code examples that apply the vocoder to a test sound, and one or more sounds that illustrate the effect of the vocoder. Another student should be able to use the vocoder given your final submission.

A List of Project Ideas

Feel free to ask the instructor for more details on any of these.

Nyquist-related

Extending Nyquist by writing functions in Nyquist:

	Port instruments from articles or other systems
	Develop library of effects with examples and documentation
	Spectral features as control parameters for synthesis. What does this mean? Many interesting forms of synthesis are based on the analysis of one sound to obtain data to control another. In this case, spectral features like brightness (the average frequency weighted by amplitude), flux (derivative of spectral amplitude), and frequencies of peaks (also known as peak-tracking) can be used to obtain control information. A project would consist of some analysis software to extract spectral features and some compelling examples of sound synthesis using those features.
	Implementation or ports of spectral manipulation software (e.g. from Eric Lyon and Chris Penrose)
	Rythm guitar synthesizer that takes a strumming pattern, a chord progression, and a guitar string synthesizer function and computes a sound. This should make it possible to create a range of guitar performances without having to notate the time and pitch of each note.
	Tom Neuendorffer and Roger Dannenberg recently recorded Tom's hammered dulcimer using both wood and felt hammers and using single and multiple bounce playing. An interesting project would be to trim the recordings to keep only the good stuff, identify the note onsets and durations, build a hammered dulcimer instrument in Nyquist that pulls in and mixes samples from disk, possibly add some "extended dulcimer" functions like vibrato, pitch bend, an extended range, and string damping using envelopes, and create a demo piece with some (synthetic) virtuoso hammered dulcimer playing.

Extending Nyquist by writing functions in C

	Phase Vocoder support
	Physical models from Perry Cook (partially implemented)
	MQ analysis/synthesis (or the more advanced Loris system or Juan Pampin's system)
	Support VST plugins in Nyquist.
	Allow Nyquist to import csound unit generators
	Support SDIF I/O (see SDIF library from UCB)
	Add support for Allegro score representation
	I have some code from a previous project to read Gravis Ultrasound Sound Font files. The goal is to be able to import sounds, envelope data, etc. from these files to construct a Nyquist function that can play instruments described by these files. This would instantly expand Nyquist's library of conventional instrument sounds.

Extending the Nyquist IDE by writing functions in Java

	Add GUI controls to Nyquist. Nyquist already has an array of floats that can be accessed at run time to get scalar (float) values or time-varying signals. These "sliders" can be set by sending special strings from the IDE or by sending OSC messaages over the network. What is lacking is some Java IDE code to create sliders, save configurations in the workspace.lsp file, etc. Probably slider creation should be done by graphical interaction, but the configuration should be saved in the workspace (see the envelope editor and EQ editors for examples).
	The Nyquist IDE has plenty of rough edges. I don't like to encourage "cosmetic coding" as a class project, but if you are an expert Java GUI programmer, there are a number of problems that could use your skills: The window layout is different on every system, but there must be a way to do a good job of window sizing and placement. Fonts are terrible on Linux. The sliders used in the Browser and other places are not compact -- it would be nice to have sliders that look like those in professional audio applications. Parenthesis balancing has come a long way from the original implementation, but it could still use some improvement. In particular, when the cursor is positioned after a close parenthesis, the corresponding open parenthesis should be highlighted. Graphical object event handling in the code is inconsistent. A "design pattern" for creating and handling graphical objects to guide future implementation and rewrites would be nice.
	The EQ editor is preliminary -- it should allow the user to specify the range of frequencies, the number of frequency channels, and support multiple audio channels, either locked together or independent.
	The envelope editor is awkward, lacks labels, and editing features, but it's a good start. A good project would propose an improved design and implement a replacement.
	The IDE's "piano roll" score editor is a rough start, and the user interface needs a lot of work, including scroll bars, labels, zoom, selection, access to attributes other than pitch and time, the ability to quantize or snap to grids, etc. A project in this area needs to make a specific interface design and implementation plan.
	Spatial positioning of sound is difficult to do in Lisp. A graphical editor might help to position sound sources in stereo or 5.1 or whatever. Especiallly interesting would be the ability to specify moving sound sources and to edit their paths.
	There have been several attempts to create a graphical score entry system. The idea is that you represent different Nyquist behaviors with different shapes and/or colors on a pitch vs. time graph. This is similar to a piano-roll editor except the shapes or colors indicate different behaviors (Nyquist functions). There may be other parameters encoded in the height, color, texture, or shape of each graphical object in the score. Each object can be edited using a property sheet, and all the properties are passed to Nyquist as parameters to the behavior, and behaviors are organized in terms of time and duration according to their (editable) graphical layout. The main idea is to provide some graphical score editing but to avoid strong ties to traditional music notation or assumptions that music is best represented in terms of pitch and time (only). Maybe this should be a superclass of the piano roll editor or the two should share an abstract superclass.
	Nyquist has some nice drum samples, but the interface for creating drum patterns is very primitive. On the other hand, it's not easy to create a good graphical interface to edit drum patterns. It is especially important to support unconventional patterns, odd meters, etc., but not obvious how to do this in a user-friendly way.
	Speaking of drum patterns, Dan Trueman demonstrated a fascinating interface for rhythmic patterns called the Cyclotron (ICMC 2008). Although his system was real-time and interactive, his original work was a non-real-time system. Given Nyquist's quick computation, I think it would interesting to create a Cyclotron-like interface in Java that outputs Nyquist scores.

Audacity-related -- In general, students have had a difficult time getting up to speed and contributing to Audacity, but at the same time, there is much room for improvement

Audacity uses an old version of Nyquist. An update would be non-trivial because of the number of changes to Nyquist. Audacity requires special modifications to disallow operations that could be used to attack a user through malicious plug-ins (e.g. file-io). The Audacity/Nyquist interface is not quite right -- it does not allow access to all of Audacity, making it hard or impossible to write plug-ins that utilize more than one track. Also, the selection can only be read once in Nyquist, making it impossible to do things like normalization without storing the entire audio selection in Nyquist memory.

Scales to display time, beats, samples, etc.

Add breakpoint envelope editor to Audacity

Ability to edit a tempo track and display beat locations

Add MIDI editing functions (requires better event representation)

Audacity editing interface for editing live recordings (consult with Riccardo Schultz)

	cross-fade editor dialog
	play list interface
	play list engine to compute and play segments and crossfades

Tutorials/Presentation

Develop animations, diagrams, and sounds to illustrate course concepts, such as:

Sampling theory

Synthesis methods

FFT

If you do something like this, you should be sure to get the content and presentation method approved.

Other

	Develop open-source beat-tracking software.
	Module to send digital audio over networks
	Add support for SDIF I/O to Audio File Library
	Build system to segment a performance into individual notes
	Play synchronized midi and audio using PortMidi and PortAudio
	Develop/evaluate software to track beats, making progress toward a system for beat-synchronous digital audio effects for pop/rock bands. In general, robust tempo and beat tracking does not exist, but there are some good techniques for extracting beat features and a lot of work has been done. (By the way, amplitude peaks are an obvious feature, but do not work well in general.) For a specialized application such as tracking beats in a rock band, there are many "tricks" such as putting a mic on the bass drum or bass guitar, looking for drum patterns, and even hooking up to MIDI output from a keyboard (if any). Trying to build a good beat tracking system from scratch is not a good idea, but evaluating a specialized approach to a narrowly defined problem, building upon known techniques would be very interesting.
	Work on sound file support in Nyquist. I recently adopted libsndfile, but then found that FLAC support is apparently not up-to-date (there are other libraries for FLAC) and as mentioned above, there is no support for SDIF files. It would also be cool to be able to read/write mp3 files directly. I think the ability to read loop data out of AIFF sample files has been lost in the conversion to libsndfile, and I'd like to put it back -- ideally offering improvements to libsndfile. Finally, libsndfile does not really support compilation to universal binaries on the Mac or compilation under Win32. I hacked the sources to make this work, but now libsndfile (in nyquist/nylsf) is dependent on a nyquist configuration file. This whole thing needs some thought -- how to incorporate various libraries in a cross-platform way without having to install gnu tools and run config for every library on every platform.