\section{Existing Research Framework}
\label{nespole}

The proposed \spectrum project builds on a long-term existing collaboration
between our group at Carnegie Mellon and the three European research groups at
ITC-irst in Italy, UJF in France and the University of Karlsruhe in Germany.
We have been working closely together for the past seven years within the
C-STAR consortium\footnote{See the consortium web-site at {\tt
http://www.c-star.org\/}} and in the past 18 months have been jointly carrying
our the \nespole project.  The \spectrum project aims to further this
collaboration with a new focus on communication robustness in multi-lingual
speech-to-speech communication in wireless environments.  This section briefly
describes the {\sc Nespole!} framework that we will be building on.

The main goal of the \nespole project is to advance the state-of-the-art of
speech-to-speech translation in a real-world setting of common users involved
in e-commerce applications.  The speech-to-speech translation approach builds
on previous work of the C-STAR consortium.  The prototype system developed in
\nespole is intended to provide effective multi-lingual speech-to-speech
communication between Italian travel agents and German, French and English
clients within broad, but yet restricted domains.  The first showcase,
currently in final stages of development, is in the domain of tourism and
travel information.

\subsection{The \nespole Architecture}

\begin{figure*}[t]
%\centering
\centerline{\psfig{file=spectrum.xfig.ps,height=3in,angle=270}}
%\centerline{\psfig{file=nespole-arch.ps,height=3in}}
%\resizebox{5in}{5in}{
%\includegraphics{nespole-arch.ps}
%}
\caption{General Architecture Infrastructure for \nespole and \spectrum}
\label{nespole-arch}
\end{figure*}

The \spectrum Project will build upon the existing architectural
infrastructure developed in the course of the \nespole Project.  A main
feature of this architecture is the clean separation of the speech-translation
components of the system from the actual communication system between the two
parties participating in the dialogue.  The speech translation sub-system is
organized into language-specific Human Language Technology (HLT) servers that
can be physically distributed and that communicate with each other over the
internet.  Communication between the client and agent on the other hand is
facilitated by a dedicated module --- the {\em Mediator}.  In {\nespole}, the
mediator is designed to connect the parties using standard video-conferencing
applications.  In {\spectrum}, a new mediator module will be developed to
support the wireless communication between the mobile device on the client
side and the stationary apparatus on the agent side.  We believe that the
architecture of the HLT servers will not require any significant
modifications.  Figure~\ref{nespole-arch} shows the general architecture  
of the two systems.

Each language-specific HLT server consists of an {\em analysis chain} and a
{\em generation chain}.  The analysis chain receives an audio stream
corresponding to a single utterance and performs speech recognition followed
by parsing and analysis of the input utterance into the interlingua
representation (IF).  The interlingua is then transmitted to a central HLT
communication switch (the CS) that forwards it to the HLT servers for the
other languages as appropriate.  IF messages received from the central
communication switch are processed by the generation chain.  A generation
module first generates text in the target language from the IF.  The text
utterance is then sent to a speech synthesis module that produces an audio
stream for the utterance.  The audio is then communicated externally to the
mediator in order to be integrated back into the communication stream between
the two parties.  The system also supports the display of paraphrases
generated from the interlingua back into the language of the original speaker,
which is very useful for verification of accurate translation.

% If the paraphrase is wrong, it is likely that the produced IF was
% incorrect, and thus the translation would also be wrong.  The user may
% then use a button on the application interface to signal that the last
% displayed paraphrase was wrong.  This action triggers a message that
% is forwarded by the mediator to the other party, indicating that the
% last displayed translation should be ignored.  We will investigate the
% extent to which such functionalities can be preserved within the
% limitations of the mobile devices used for the \spectrum project.

