B3.	Objectives
The broad technological advances in computing and networking infrastructure in recent years has opened up the opportunity of new and exciting applications for the broad general public.  Among these, Electronic Commerce (E-commerce) has emerged as the application with the greatest utility for everyday general usage, consequently resulting in enormous growth over the last few years, with expectations of orders of magnitude further growth in the upcoming decade.  E-commerce opens the possibility of selling goods and services on a global market supported by the World-Wide Web. Up to now, however, interactions have mainly been menu based, guiding    potential customers through a pre-designed choice space allowing only for a limited set of predefined results. This reflects a concept for e-commerce/service that is basically driven by the supplier side.  Identification of customer requirements, tailoring existing products to the needs of a consumer and continuing maintenance of the products/services and the relationship between supplier and customer are all inherently unsupported.
A different perspective is to view e-commerce as a collaborative activity that is bi-directional and involves interaction between buyer(s) and seller(s). In such a scenario, the customer and the provider interaction directly, human to human.  They are involved in presentation, relationship building, negotiation, recommendation and collaboration, processes by which the customer is allowed to explore alternative solutions, dynamically adapting his/her goals, while the e-service/commerce provider reacts by eliciting unexpressed needs and configuring viable alternatives.  Humans can fully and naturally understand other humans’ motivations and desires, can respond to their needs in creative ways, possibly without fully matching the customer’s requests, but always trying to find solutions that anticipate and maximize the customer’s needs vis a vis concrete possibilities.
In order to make e-commerce capable of supporting negotiation between humans, the underlying technology must be capable of dealing with human-to-human communication.  Furthermore, an electronic market’s reach naturally transcends national boundaries and requires that linguistic boundaries must be removed as well.  Transnational multilingual human-to-human communication must be provided to enable flexible and effective e-commerce world-wide.  E-commerce/service provider can partially solve this problem by using teams of  human operators covering a variety of relevant languages, but such solutions would be expensive.  It would also prove unfeasible in the context of maintenance, trouble shooting and repair, where the relevant operator must possess both technical and language competence.   Resorting to a lingua franca, such as English, is only a poor substitute and naturally disenfranchises the providers from their markets and customers. 

The goal of the NESPOLE! project is to provide a system capable of supporting advanced needs in e-commerce and e-service by resorting to automatic speech-to-speech translation (henceforth, STST).  The projects builds on years of research already under way under various national and international research projects, and yet forms a departure from these traditional efforts.  Rather than focusing on automatic recognition and translation problems alone, we now focus on cross-lingual communication between humans.  The difference is that we do not necessarily only care about accuracy of translation, but the ability of two humans to communicate ideas, concepts, thoughts and to jointly solve problems.  NESPOLE! will provide a pallet of features by which this can be achieved.  Beyond mere translation in the traditional sense, NESPOLE! will feature multiple synergistic interpretation assistants (multi-engine MT approach) that range from concept driven analysis, interlingua based translation modules, over simple dictionary and phrase-book lookup to statistical direct translation schemes.  NESPOLE! will also provide for non-verbal communication by way of multimedia presentations, shared collaborative spaces and multimodal interaction and manipulation of the objects in these joint e-commerce spaces.  Enabling effective cross-lingual spoken human-human communication in the context of multimodal interaction in a joint workspace, is the overarching challenge of this project. 
The STST and multimedia technologies developed in the course of the project will be demonstrated by means of two showcases.  The first one, at the end of the first year, will support multilingual negotiations and discussion between a touristic information or service provider and a customer who wants to organize a trip exploring all available possibilities, including travel, accommodation, attractions and recreation, cultural events, dining and so on.  In a second showcase, the touristic scenario will be enlarged. In addition, a second, completely different domain will be addressed, consisting of a video help-desk for technical support in trouble shooting and repair, also capable of acting as a training environment for local retailers and technicians of an international company.  Four languages will be considered: (American) English, German, French and Italian. 

NESPOLE! intends to provide technologies addressing multilingual human-to-human communication in a multimedia environment. We will do so by improving over current experimental STST systems, addressing a number of scientific and technological issues — namely, robustness, scalability, cross-domain portability, improved translingual communication effectiveness and multimodal interaction with multimedia content.
Robustness
In order to be able to fully support  natural interaction between the partners, the system must be able to cope with the disfluencies of spontaneous speech including interruptions, corrections, repetitions, false starts, etc.. In these respects, an essential feature of the system will be its robustness — that is, the capability of dealing with corrupted inputs, due to either the peculiarities of the input utterance or to errors of the acoustic recognizer, and with incomplete information. Robustness is so important to speech-to-speech translation system that it is usual practice, in the translation process, to sacrifice fidelity to the input utterance in favour of a smooth continuation of the communication process. Thus, the system is required to be able to properly translate at least all the input content which is relevant to the accomplishment of the domain task(s), this way simplifying the goal with respect to a complete and precise translation of the input utterance.
Robustness will be assessed, both for the showcases and for the single speech/language modules, by using real speech data. At the level of a single module A, we will measure its performance, estimating A’s sensitivity to errors produced by preceding modules, this way evaluating how much A cooperates to secure robustness.  The robustness of performance of the showcases will be assessed by the users.
Multimodality
In order to be really effective  for negotiation in an e-commerce and e-service scenario, the system cannot be limited to handle speech, language or text, only. E-commerce/service already heavily relies on images, graphics and animations describing the products. The system we envision will provide multimodal support for negotiation, by allowing a close integration of and interaction between speech-based communication and visual cues and content. This means that both the customer and the provider will be able to talk about content that is presented as images or pictures and make reference to it.  Speech translation must resolve the reference that the two partners make to visually presented material.  At the same time, the systems will be able to provide supporting information dynamically, as the human-human dialog unfolds.  Many of these goals are still research topics in human-machine communication scenarios. They are particularly challenging in a translingual human-human communication setting, where multimodal coordination and synchronization must be maintained during language interpretation. Selected features of multimodality will be addressed and then demonstrated in the implemented showcases. 
Scalability and cross-domain portability
Present speech-to-speech translation systems are highly domain dependent.  In order to improve effectiveness and performance, both the linguistic resources and many of the speech/language engines are built around a particular domain at hand. This results in systems which do not easily scale up, and which are difficult to port from one domain to the other without major changes to both linguistic resources and engines. Although attempts at addressing scalability and cross-domain portability have already been made in many areas of HLT, these are rather new concerns in speech-to-speech translation.  They are crucial, though, to offer speech-to-speech translation as a viable technological solution in many domains. The NESPOLE! project aims to explore methodologies for dealing with such problems by exploring multi-engine STST systems; improving the performances of single engines architectures; enhancing existing Intermediate Representation Formalism (IRF) to provide for more flexible, easy to use and to extend domain and meaning encoding; using Information Extraction techniques to (semi-) automate the process of domain knowledge acquisition.
The results achieved with respect to scalability will be demonstrated by enlarging the tourism domain of the first showcase into a more complete one, demonstrated by the second showcase. Similarly, cross-domain portability will be demonstrated by porting the system to a second domain — namely, the video help-desk domain.
Improved translation quality
E-service and e-commerce scenarios require high quality translation, especially with respect to such issues as dialogue structure, lexical choices, topic/focus articulation, etc. NESPOLE! intends to explore these issues and implement in the showcases the relevant results.
Awareness with respect to technological evolutions
NESPOLE! will take care of the viability of the proposed solutions with respect to the underlying telecommunication technologies and platforms. This means that the project does not only aim at showing the possibility of successfully integrating the results with state-of-the-art technologies, but will also attempt at securing the compatibility with foreseeable evolutions. A crucial role in securing the achievement of such objectives will be played by the technological partner.