\section{Scientific Goals and Objectives}

The focus of the proposed \spectrum project is communication
robustness in spoken language translation (SLT).  In addition to
conducting scientific research to advance the state-of-the-art in SLT,
we will build a prototype of a mobile platform for travel related
services.  The platform will include spoken language translation for
human-human communication as well as spoken human-machine dialogue for
other information and services.  Both spoken human-human communication
and human-machine communication will be enhanced with fall-back
modalities to smooth over communication failures.  Evaluation will be
based on a suite of metrics related to usability.

The context of the research is the continuing progress of the
computing revolution toward mobile, wearable, and ubiquitous devices.
Wireless devices and increased bandwidth provide an enormous spectrum
of possibilities for communication, provided that content and services
can be brought to the user in a contextually appropriate and natural
manner. \spectrum aims to exploit the emerging power of mobility to
better serve cross-cultural and cross-linguistic communication and
computing needs.

\spectrum builds on our previous research in the {\sc Nespole!}
consortium~\footnote{{\it Nespole\/} ({\tt NES-po-lay}) means literally {\it
loquat fruit\/} in Italian.  As an exclamation, it means {\it Wow!\/}.  See
the project web-site at {\tt http://nespole.itc.it\/}}.  However, it goes
beyond {\sc Nespole!} in addressing a new round of scientific challenges and
in focusing on a broader notion of communication robustness that goes beyond
the noise tolerance of parsing and speech recognition.

Most current efforts in spoken language translation address a rather
narrow notion of robustness, targeting the capability of modules and of the
system as a whole to provide sensible answers even in the presence of
corrupted input, incomplete information, etc. Such a concern is natural as
long as SLT systems play a passive role, and their objective is limited to
translation of isolated messages from one language to another. For actual use,
though, it is crucial to start addressing the question of how SLT systems can
be designed to play a more active role in securing achievement of
communicative goals.  Communication robustness encompasses, in our view, all
the ways in which communication can fail: communicative goals misalignments,
misunderstandings among the parties, problems due to the limitations of the
system itself (e.g., HLT modules), or of the underlying networking, etc.

We identify the following facets of communication robustness: tolerance to
noise; adaptability and learning; detection of communication failure; seamless
integration of fall-back modalities for repair; seamless integration of
services using different modalities (e.g., human-human translation and
machine-human navigation assistance); and absorbing information from more
modalities (e.g., prosody and information about the emotion of the speaker).
Also, we are moving from evaluations of translation accuracy to evaluations of
system usability.  Each of the new scientific challenges of \spectrum
addresses one of these desiderata.  Following is a list of the research areas
of \spectrum with indications of how they contribute to communication
robustness.

\begin{itemize}
\item In speech recognition: tolerance to
environmental noise, speaking rate, stress and emotion ({\it
addressing noise tolerance\/}); new work on the optimal integration of
multilingual recognizers ({\it addressing adaptability and
learning\/}); and detection of emotions ({\it addressing absorbing
information from more modalities\/}).

\item For machine translation, we will maintain our interlingua
approach because of its usefulness in multilingual situations.
However we propose two new automated learning techniques to enhance
the human-engineered language analyzers ({\it addressing adaptability 
and learning\/}), and a new model of multi-engine integration ({\it
addressing seamless fall-back\/}).

%% new direct translation methods?

\item A new dialogue monitor will flag communication failures ({\it
addressing detection of failure\/}).  The 
core of the dialogue monitor will be new confidence measures for the
language analysis components and analysis of speech prosody ({\it
addressing absorbing information from more modalities\/}).

\item A new modality manager will facilitate the communication
process.  The modality manager will facilitate between several human
agents via speech translation as well as with computer agents that
supply complementary information or requested data.  Taking cues from
the dialogue monitor, it will suggest fall-back modalities when
communication failures are detected ({\it addressing seamless
fall-back and integration of services using different modalities\/}).

\item A flexible design of the telecommunication architecture will be 
undertaken, with careful attention given to the potential and
limitations of foreseeable networking solutions. We will propose
architectures that can be actually deployed in the medium term, but
expanded as more advanced communication systems become available.

\item Our methodology for sentence-level accuracy-based evaluations
will be augmented by new techniques for usability, including achievement of
communicative goals, platform adequacy (footprint and bandwidth), and
optimality of selecting and integrating different services (such as
translation and navigation).
\end{itemize}

\spectrum is designed to be a joint collaborative project between our group 
at the Language Technologies Institute at Carnegie Mellon and three European
research groups at ITC-irst in Italy, Universit\'{e} Joseph Fourier in France
and the University of Karlsruhe in Germany.  The project consortium will also
include Aethra - an Italian tele-communications commercial company.  It
follows on the footsteps of the currently in progress \nespole project
involving the same consortium members.  \nespole is funded jointly by the
European Commission Fifth Framework (funding the European groups) and the NSF
MLIAM Program (funding our group at CMU).  A parallel proposal for the
\spectrum project has been submitted to the EC (IST-2001-34738), and is
appended as an addendum to this document.  This proposal submitted to the
NSF-ITR Program requests funding for Carnegie Mellon's participation in the
project.

In this proposal we focus specifically on the core scientific research issues
for which our group at Carnegie Mellon will have primary responsibility.
Additional research issues for which the European partners have primary
responsibility are described in greater detail in the proposal submitted to
the EC.  It is our belief that the investigation of these core research
questions identified above can be carried out to a large extent independently,
in case European funding is not secured.  We believe the research described
clearly fits under the ITR focus area of ``Augmenting Individuals and
Transforming Society'', particularly as it aims to design and develop a
multi-lingual system ``to serve the needs of multi-national industry,
collaborating science teams, or virtual cultural exchanges''.
