Getting a computer music system to listen to a live performer and synchronize an existing accompaniment.
See also Music Understanding
Back to Bibliography by Subject
Dannenberg, ``An On-Line Algorithm for Real-Time Accompaniment,'' in Proceedings of the 1984 International Computer Music Conference, Computer Music Association, (June 1985), 193-198.
PDF Version.This was the first of my papers on computer accompaniment. It presents algorithms for following a monophonic soloist in a score and synchronizing the accompaniment.
Dannenberg, Method and apparatus for providing coordinated accompaniment for a performance. US Patent #4745836, 1988.
The first computer accompaniment patent. The patent describes monophonic and polyphonic accompaniment systems.
Buxton, Dannenberg, and Vercoe, ``The Computer as Accompanist,'' in CHI '86 Conference Proceedings (April 1986), ACM/SIGCHI, pp. 41-43.
A short paper (actually presented as a panel discussion) with statements from me, Barry, and Bill.
Bloch and Dannenberg, ``Real-Time Accompaniment of Polyphonic Keyboard Performance,'' in Proceedings of the 1985 International Computer Music Conference, (August 1985), pp. 279-290.
PDF Version.Expands previous work with 2 algorithms for matching polyphonic (keyboard) to a stored score, and also describes various techniques for making accompaniment timing more musical.
Real-Time Scheduling
Dannenberg and Mont-Reynaud, ``Following an Improvisation in Real Time,'' in Proceedings of the International Computer Music Conference, Computer Music Association, (August 1987), pp. 241-248.
Expands previous work by addressing the problem of following solos improvised over fixed chord progressions rather than fixed note sequences. New beat-tracking and matching algorithms are introduced.
Dannenberg and Mukaino, ``New Techniques for Enhanced Quality of Computer Accompaniment,'' in Proceedings of the International Computer Music Conference, Computer Music Association, (September 1988), pp. 243-249.
Introduces the idea of using multiple matchers centered at different locations to make following more robust. Also deals with trills, glissandi, and grace notes, which cannot be matched note-for-note.
ABSTRACT:A computer accompaniment system has been extended with several techniques to solve problems associated with reliably recognizing and following a real-time performance. One technique is the use of an additional matcher to consider alternatives when the performance could be interpreted in two ways. Another is to delay corrective action when data from the matcher is suspect. Methods for handling grace notes, trills, and glissandi are also presented. The implementation of these and other techniques to produce an enhanced computer accompaniment system is discussed.
[Adobe Acrobat (PDF) Version.] [Postscript Version.]
Grubb and Dannenberg, ``Automated Accompaniment of Musical Ensembles,'' in Proceedings of the Twelfth National Conference on Artificial Intelligence, AAAI, (1994), pp. 94-99.
Grubb and Dannenberg, ``Automating Ensemble Performance,'' in Proceedings of the 1994 International Computer Music Conference, International Computer Music Association, (September 1994), pp. 63-69.
Grubb and Dannenberg, ``Computer Performance in an Ensemble,'' in 3rd International Conference for Music Perception and Cognition Proceedings, European Society for the Cognitive Sciences of Music, Liege, Belgium, pp. 57-60, 1994.
These three papers discuss the problem of score following when there are multiple players. At any given time, some players may not be playing, some may be lost, and others are providing good information. These papers cover similar material in different levels of detail and for different audiences. Probably the ICMC paper is best, although the AAAI paper assumes a more CS and less musical background.
Dannenberg, Automated musical accompaniment with multiple input sensors. US Patent #5521324, 1994.
This patent is related to the ensemble accompaniment research reported in the papers mentioned above. Here is the abstract: The present invention is directed to an apparatus and method for automating accompaniment to an ensemble's performance. The apparatus is comprised of a plurality of input devices with each input device producing an input signal containing information related to an ensemble's performance. A plurality of tracking devices is provided, with each tracking device being responsive to one of the input signals. Each tracking device produces a position signal indicative of a score position when a match is found between the input signal and the score and a tempo estimate. A first voting device is responsive to each of the position signals for weighting each of the position signals. The weighting may be based on the frequency with which it changes and the proximity of its score position to each of the other score positions represented by each of the other position signals. The same weighting factors are then applied to the tempo estimate associated with that position signal. After the position signals and tempo estimates have been weighted, the voter device calculates a final ensemble score position signal in response to the weighted position signals and a final ensemble tempo based on the weighted tempo estimates. A scheduler is responsive to the final score position and final ensemble tempo for outputting an accompaniment corresponding thereto.
Grubb and Dannenberg, ``A Stochastic Method of Tracking a Vocal Performer,'' in Proceedings of the International Computer Music Conference, San Francisco: International Computer Music Association, (1997), pp 301-308.
Lorin Grubb's Ph.D. thesis introduced a completely new approach to score following in which the position in the score is represented by a probability density function. This and the following paper report on this work, but you should also read Lorin's Ph.D. thesis (available from CMU SCS) for the whole story.
Abstract. Automated accompaniment systems are computer systems that "play along" with a solo musician or a group of musicians when given the score for a piece of music. These systems must be able to "listen" to the live musicians by tracking their progress through the score in real-time. Accurate tracking of vocal performers (as opposed to instrumentalists) is a particularly challenging case. In this paper, we present a tracking method based upon a statistical model of vocal performnances. This technique incorporates both information obtained from real-time signal processing of a performance (such as fundamental pitch) and information describing the performer's movement through the score (namely tempo and elapsed time). We present a description of how this model is incorporated as part of a system currently used to accompany vocal performers. We provide a preliminary evaluation of its ability to estimate score position of a performer.
[Adobe Acrobat (PDF) Version.]
Grubb and Dannenberg, ``Enhanced Vocal Performance Tracking Using Multiple Information Sources,'' in Proceedings of the International Computer Music Conference, San Francisco: International Computer Music Association, (1998), pp. 37-44.
Abstract. We describe an enhanced statistical method for tracking vocal performances in real time. This approach integrates multiple measurements of the sound signal that are relevant to determining score position. The information is combined using a statistical model that incorporates distributions estimated from actual vocal performances. We present results from using different combinations of signal measurements to track both recorded and live performances. Results of this testing indicate some requirements for successfully using multiple information sources to improve automated vocal performance tracking.
[Adobe Acrobat (PDF) Version.]
Grubb and Dannenberg, System and method for stochastic score following. US Patent #5913259, 1997.
This patent is related to the vocal accompaniment work described in the paper mentioned above and in Lorin's thesis. Here is the abstract: The present invention is directed to a computer implemented method for stochastic score following. The method includes the step of calculating a probability function over a score based on at least one observation extracted from a performance signal. The method also includes the step of determining a most likely position in the score based on the calculating step.
Dannenberg and Raphael, ``Music Score Alignment and Computer Accompaniment,'' in Communications of the ACM, Volume 49, Number 8 (August 2006), pp. 38-43.
Abstract. We focus here on a form of machine listening known as music score matching, score following, or score alignment. Here we seek a correspondence between a symbolic music representation and an audio performance of the same music, identifying the onset times of all relevant musical ``events'' in the audio -- usually notes. There are two different versions of the problem, usually called ``off-line'' and ``on-line.''
[Adobe Acrobat (PDF) Version.]