Interactive Computer Music Systems
A session of the Acoustical Society of
America (ASA) conference in Pittsburgh, PA,
Tuesday Morning, 8:25-11:30am, June 4, 2002,
chaired by Roger B. Dannenberg,
of Computer Science, Carnegie Mellon University,
This session explores computer music performance systems and virtual
instruments, especially the use of sensors and computing to control real-time
sound generation processes. The presenters are an internationally distinguished
collection of scientists, composers, and musicians, and most are confortable
in two or more of these roles.
8:25 Chair's comments: Roger Dannenberg
The structural implications of interactive creativity
Joel Chadabe, Electronic Music Foundation, Ltd.
The functioning of any particular electronic musical instrument can be
placed somewhere along a line that extends from deterministic to indeterministic.
Bearing in mind this author's conviction that the goal of technology should
be to better human existence, we ask: In what ways does an electronic musical
instrument function for the benefit of its performer? Although deterministic
instruments may offer more powerful controls than traditional instruments,
they typically put a performer in the traditional situation of making a
gesture and expecting a predictable effect. Indeterministic instruments,
on the other hand, put a performer in an interactive role of improvising
relative to an unpredictable output. By 'interactive', I mean 'mutually
influential'. The performer influences the instrument and the instrument
influences the performer. The unique advantage of such interactive instruments
is that they foster 'interactive creativity'.
The design of a traditional instrument is fundamentally different from
the design of an interactive instrument. A traditional instrument
is structured as a single cause and effect, articulated as a synchronous
linear path through a hierarchy of controls from a performer operating
an input device to the multiple variables of a sound generator. An interactive
instrument, on the other hand, is structured as a network of many causes
and effects at various levels of importance, with a performer's input as
only one of the causes of the instrument's output in sound.
The author will present several historical examples of interactive electronic
musical instruments and offer some speculations on the future.
Music scene description: Toward audio-based real-time music understanding
Masataka Goto, PRESTO, JST. / National Institute of Advanced Industrial Science and Technology (AIST, Japan)
Music understanding is an important component of audio-based interactive
music systems. A real-time music scene description system for the
computational modeling of music understanding is proposed. This research
is based on the assumption that a listener understands music without deriving
musical scores or even fully segregating signals. In keeping with this
assumption, our music scene description system produces intuitive
descriptions of music, such as the beat structure and the melody and bass
lines. Two real-time subsystems have been developed, a beat tracking subsystem
and a melody-and-bass detection subsystem, which can deal with real-world
monaural audio signals sampled from popular-music CDs. The beat tracking
subsystem recognizes a hierarchical beat structure comprising the quarter-note,
half-note, and measure levels by using three kinds of musical knowledge:
of onset times, of chord changes, and of drum patterns. The melody-and-bass
detection subsystem estimates the F0 (fundamental frequency) of melody
and bass lines by using a predominant-F0 estimation method called PreFEst,
which does not rely on the F0's unreliable frequency component and obtains
the most predominant F0 supported by harmonics within an intentionally
limited frequency range. Several applications of music understanding are
described, including a beat-driven, real-time computer graphics and lighting
Making the computer ``listen'' to music
Christopher Raphael, Dept. of Mathematics and Statistics,
University of Massachusetts, Amherst
A computer system is discussed that provides real-time accompaniment to
a live musician playing a non-improvisatory piece of music. Particular
attention is devoted to the ``listening'' process, in which the computer
must follow the soloist's progress through the musical score by interpreting
the sampled acoustic signal. The process is complicated by the significant
variation and occasional errors from the live player during performance.
A hidden Markov model is introduced providing a principled, trainable and
fast solution to the listening problem. The system is capable of
assessing its own level of uncertainty about score position, as well as
accomodating the sometimes strong signal component from the accompaniment
instrument. A live demonstration will be provided.
Robert Rowe, Department of Music and Performing Arts Professions,
School of Education, New York University
The training of musicians begins by teaching basic musical concepts, a
collection of knowledge commonly known as musicianship. Computer programs
designed to implement musical skills (e.g., to make sense of what they
hear, perform music expressively, or compose convincing pieces) can similarly
benefit from access to a fundamental level of musicianship. Recent research
in music cognition, artificial intelligence, and music theory has produced
a repertoire of techniques that can make the behavior of computer programs
more musical. Many of these were presented in a recently published book/CD-ROM
entitled “Machine Musicianship”. For use in interactive music systems,
we are interested in those which are fast enough to run in real time and
that need only make reference to the material as it appears in sequence.
This talk will review several applications that are able to identify the
tonal center of musical material during performance. Beyond this specific
task, the design of real-time algorithmic listening through the concurrent
operation of several connected analyzers is examined. The presentation
includes discussion of a library of C++ objects that can be combined to
perform interactive listening and a demonstration of their capability.
The IMUTUS interactive music tuition system
George D. Tambouratzis, Stelios Bakamidis, Ioannis Dologlou, George Carayannis,
Markos Dendrinos, Institute for Language and Speech Processing, Greece
This presentation focuses on the IMUTUS project, which concerns the creation
of an innovative method for training users on traditional musical instruments
with no MIDI (Musical Instrument Digital Interface) output. The entities
collaborating in IMUTUS are ILSP (co-ordinator), EXODUS, SYSTEMA, DSI,
SMF, GRAME and KTH.
The IMUTUS effectiveness is enhanced via an advanced user interface
incorporating multimedia techniques. Internet plays a pivotal role during
training, the student receiving guidance over the net from a specially-created
teacher group. Interactiveness is emphasised via automatic-scoring tools,
which provide fast yet accurate feedback to the user, while virtual reality
methods assist the student in perfecting his technique. IMUTUS incorporates
specialised recognition technology for the transformation of acoustic signals
and music scores to MIDI format and incorporation in the training process.
This process is enhanced by periodically enriching the score database,
while customisation to each user's requirements is supported.
This work is partially supported by European Community under the Information
Society Technology (IST) RTD programme. The authors are solely responsible
for the content of this communication. It does not represent the opinion
of the European Community, and the European Community is not responsible
for any use that might be made of data appearing therein.
Interactive systems research at CNMAT
David Wessel, CNMAT, UC Berkeley
A live-performance musical instrument can be assembled around current lap-top
computer technology. One adds a controller such as a keyboard or other
gestural input device, a sound diffusion system, some form of connectivity
processor(s) providing for audio I/O and gestural controller
input, and reactive real-time native signal processing software.
A system consisting of a hand gesture controller; software for gesture
analysis and mapping, machine listening, composition, and sound synthesis;
and a controllable radiation pattern loudspeaker are described. Interactivity
begins in the set up wherein the speaker-room combination is tuned with
an LMS procedure. This system was designed for improvisation.
It is argued that software suitable for carrying out an improvised musical
dialog with another performer poses special challenges. The processes
underlying the generation of musical material must be very adaptable, capable
of rapid changes in musical direction. Machine listening techniques
are used to help the performer adapt to new contexts. Machine learning
can play an important role in the development of such systems. In
the end, as with any musical instrument, human skill is essential.
Practice is required not only for the development of musically appropriate
human motor programs but for the adaptation of the computer-based instrument