|The VideoHarp, manufactured by SensorFrame, was origninally developed by Paul McAvinney and Dean Rubine in the Computer Music Project.
Combined Spectral Interpolation Synthesis (CSIS) Model. (Multiple sound examples with explanation.)
Concatenative Synthesis Using Score-Aligned Transcriptions (Sound examples and some explanation.)
Video example of Computer Accompaniment. (13MB) This is a good example and explanation of pioneering work in computer accompaniment conducted from 1983 to 1985, ending with this program that was featured at the Commodore Amiga product announcement. The software was developed by Roger Dannenberg, Josh Bloch, and Ralph Bellafatto (at Cherry Lane Technologies). This QuickTime video was made around 1990 using the 1985 version of the software.
Video example of Computer Accompaniment of Vocal Music. (2MB) In this video from around 1997, the singer is accompanied by a computer. The notes and other score information was entered by hand from the score. The computer listens to the singer (using the microphone you see in the video), follows her performance in the score, and synchronizes its accompaniment to her vocal performance. The piano you hear is a MIDI synthesizer. All processing is live and taking place in real time.
Video excerpt from "In Transit." (2MB) "In Transit" is an interactive composition for computer music, computer graphics, and human soloist. The computer listens to the soloist and classifies the performance according to style. The styles are "lyrical," "syncopated," "pointilistic," and "frantic." The absolute meanings of these terms is not important. What is important is that the player and computer agree so that the computer can understand what the player intends. The computer actually learns about style from examples (see Dannenberg, Thom, and Watson, “A Machine Learning Approach to Musical Style Recognition” ). In the video, there is a transition from syncopated to frantic. I intentionally made the syncopation slightly more and more frantic until the computer decided to kick in with its own frantic response. At that point, I let out all the stops. The animation is by Scott Draves and is an integral part of the piece. I watch and listen as I perform, and the computer listens to me.
The Piano Tutor. (34MB) The Piano Tutor is an intelligent piano instruction system. You can read more about it in on-line papers. This video won the best video award at CHI '90. The reference is: Sanchez, Joseph, Dannenberg, Capell, Saul, and Joseph, ``The Piano Tutor,'' in ACM SIGGRAPH Video Review Issue 55: CHI '90 Technical Video Program (video). 1990.
Sound from Video. (4MB) A novel synthesis technique is introduced where digital video controls sound spectra in real time. Light intensity modulates the amplitudes of 32 harmonics in each of several synthesized “voices.”
Automatic lyric alignment a modified speech model constructed for forced alignment of lyrics to audio using the CMU Sphinx speech recognition system.
Soundcool with Algorithmic Control. Soundcool is a creative construction kit for making and controlling sound. Our group at Carnegie Mellon has been collaborating with Soundcool. In this example, we show how some simple code can automate Soundcool module controls to produce something you could not produce by Soundcool alone.