\mysection{Scientific Challenges and Innovations}
\label{challenges}


{\sc Nespole!} intends to improve over current experimental speech-to-speech
translation systems, addressing a number of scientific and technological
issues: robustness, scalability, cross-domain portability, and multimodal
interaction with multimedia content.  The specific innovations that
will be contributed by CMU include: two new robust
parsing methods; a robust, scalable interchange format for multilingual
meaning representation in multiple domains; and the integration of the speech
translation modules in a multimodal shared workspace.

\paragraph{Robustness:} %% and Fidelity

In order to fully support natural interaction between the partners,
the system must be able to cope with corrupted inputs, due to either
the peculiarities of the input utterance or to errors of the acoustic
recognizer, and with incomplete information.  Robustness in {\sc
Nespole!} will be achieved with an array of methods including two new,
innovative parsers --- the Concept Classification parser ({\sc cc})
and the {\sc LCFlex} parser (Section~\ref{translation}) --- and
multi-engine integration of knowledge-based and corpus-based machine
translation methods.

%% Robustness is so important to STST systems that it is usual practice
%% to sacrifice fidelity to the input utterance.  For example, as in our
%% previous work, the system may be required to translate only what is
%% relevant for accomplishing the domain task, ignoring many niceties and
%% out-of-domain portions of the source language input \cite{}.
%% {\sc Nespole!} will attempt a more equitable balance of robustness and
%% fidelity.  

%% A pilot version of {\sc cc} has been tested in the travel domain.
%% {\sc LCFlex} also exists in a preliminary version but has yet to be
%% run in a major application.  Section~\ref{translation} describes the
%% open research questions concerning these two parsing methods.

\paragraph{Scalability and Cross-Domain Portability:}
In present STST systems, robustness is achieved at the cost of
breadth.  Thus, STST systems are usually confined to narrow semantic
domains.  Although attempts at addressing scalability and cross-domain
portability have already been made in many areas of language
technologies, these are rather new concerns in STST.  They are
crucial, though, to offer STST as a viable mode of communication.  The
two {\sc Nespole!} showcase systems will demonstrate broadening STST
in two ways.  The first showcase will demonstrate scalability through
the expansion of the travel domain of our previous work
\cite{amta-98,mtj}. The second showcase will demonstrate cross-domain
portability through a new application in a video help-desk domain.

The key to scalability in {\sc Nespole!} is the distinction between
{\it task-oriented\/} and {\it descriptive\/} sentences.  (See
Section~\ref{irf}.)  Most STST systems (possibly with the exception of
\cite{frederking+al:97a,Frederking+et+al:99}) are limited to
task-oriented sentences.  However, even task-oriented dialogues
include some descriptive sentences.  Not handling these makes for a
dry and not very detailed dialogue.  We argue that the robust methods
that have been effective for task-oriented sentences are not effective
for translation of descriptive sentences, and propose methods for
translation of descriptive sentences in spoken dialogue.  Innovations
in this area include a renovation of our interlingua design to
integrate representations of task-oriented and descriptive sentences,
and a lexically-driven option in the {\sc LCFlex} parser for robust
parsing of descriptive sentences.

Cross-Domain portability in {\sc Nespole!} will be addressed by the
development of new translation approaches that are inherently more portable
than existing technology and by the integration of direct translation engines
that, while shallow in coverage, are far more domain-independent than the
proposed IF-based approaches.  The new concept classification parser that
we propose relies on an underlying phrase-level semantic grammar that is
far less domain dependent (thus far more portable) than complete semantic
grammars.  The concept classification itself will be based on data-trainable
technology (such as HMMs and neural nets), that are inherently more suitable
for portability to new domains.  Our other new translation approach will
focus on the translation of descriptive sentences, and will be designed to be
domain independent as much as possible, thus supporting portability to new
domains.

\paragraph{Multimodality:}
The integration of STST in an environment supporting multimodality and
multimedia presentation in large and complex domains goes well beyond
the capabilities of current STST systems and state-of-the-art
technologies (telephone-based call-centers, or by Web-based
e-commerce/service).  The main challenge {\sc Nespole!} will face is
the proper treatment of synchronization between language and visual
information: already a hard task, now complicated by STST. The fact
that the linguistic message goes through a translation stage means
that straightforward methods for synchronizing pointing acts and
natural language referring expression based on measurements performed
on an underlying time-line are not viable. Synchronization is a
symbolic process, which can only be captured as obtaining between
pointing acts, as performed by humans on images on the screen, and
semantic units, as encoded in IF expressions. Therefore, {\sc
Nespole!}  will provide: methods for detecting the synchronization
between acoustic events and pointing acts, methods for encoding the
synchronization at the interlingua (symbolic) level, and methods for
re-building the synchronization into the target multimodal scenario.


%% In order to be really effective for negotiation in e-commerce/service,
%% the system cannot simply handle speech, language or text in isolation.
%% E-commerce already heavily relies on images, graphics and animations
%% describing the products.  {\sc Nespole!} will provide multimodal
%% support for negotiation by allowing both the customer and the provider
%% to talk about content that is presented by means of images and make
%% reference to it, while speech translation will resolve the reference
%% that the two partners make to visually presented material.  The system
%% will also retrieve supporting information dynamically, as the
%% human-to-human dialogue unfolds.  Finally, multimodal interaction will
%% also be helpful in the communication process itself, as it
%% disambiguates language input.

%% \mysubsection{Improved translation quality}

%% E-service and e-commerce scenarios require high quality translation.
%% {\sc Nespole!} will address three issues related to translation
%% quality: dialogue structure, lexical choices, and topic/focus articulation.

%% We can't say this without, e.g., a proposal for lexical selection,
%% theory of topic/focus, etc.

%% \mysubsection{Awareness with respect to technological evolutions}

%% NESPOLE! will take care of the viability of the proposed solutions with
%% respect to the underlying telecommunication technologies and platforms. That
%% is, the project does not only aim at showing the possibility of successfully
%% integrating the results with state-of-the-art technologies, but will also
%% attempt at securing the compatibility with foreseeable evolutions.


%% \mysubsection{Innovation}

