Language Technologies Ph.D. Thesis Defense

  • Ph.D. Student
  • Language Technologies Institute
  • Carnegie Mellon University
Thesis Orals

Socially-Aware Dialog System

In the past two decades, spoken dialog systems, such as those commonly found in cellphones and other interactive devices, have emerged as a key factor in human-computer interaction. For instance, Apple's Siri, Microsoft's Cortana, and Amazon's Alexa help human users complete tasks more efficiently. However, research in this area has yet to produce dialog systems that build interpersonal closeness over the course of a conversation while also carrying out the task. This thesis attempts to address that shortcoming. Specifically, research in computational linguistics (Bickmore and Cassell,1999) has shown that people pursue multiple conversational goals in dialog, which include those that fulfill propositional functions to contribute information to the dialog; those that fulfill interactional functions to manage conversational turn-taking; and those that fulfill interpersonal functions to manage the relationship between interlocutors. Although spoken dialog systems have greatly advanced in modeling the propositional and, to a lesser extent, interactional functions of human communication, these systems fall short in replicating the interpersonal functions of conversation. We propose that this interpersonal deficiency is due to insufficient models of interpersonal goals and strategies in human communication.

As dialog systems become more common and are used more frequently, propositional content and interactional content will not suffice. In this thesis, therefore, we address these challenges by proposing a socially-aware intelligent framework that exploits a path to systematically generate dialogs that fulfill interpersonal functions.

In our work (Zhao et al., 2014), we clarify that a socially-aware intelligent framework can explain how humans in dyadic interactions build, maintain, and tear down social bonds through specific conversational strategies that fulfill specific social goals and that are instantiated in particular verbal and nonverbal behaviors. In order to operationalize this framework, we argue that four capabilities are needed to achieve a socially-aware intelligent system. The system must (1) automatically infer human users' social intention by recognizing their social conversational strategies, (2) accurately estimate social dynamics by observing dyadic interactions, (3) reason through appropriate conversational strategies while accounting for both the task goal and social goal, and (4) realize surface-level utterances that blend task and social conversation. Our socially-aware dialog system focuses on blended conversations that mix a goal-oriented task with social chat. As a proof of concept, we exploit a modular-based socially-aware personal assistant to aid conference attendees by eliciting their preferences through building rapport, and then making informed personalized recommendations about sessions to attend and people to meet.

Finally, we leverage the power of neural networks to model negotiation dialog within our socially-aware intelligent framework. We present a novel learning paradigm to formulate a two-phase computational model to blend negotiation utterances and social conversation that requires less human supervision than traditional modular-based approaches. Our comprehensive experiments show that the system can facilitate negotiation while building a social bond with a human user.

Thesis Committee:
Alexander I. Rudnicky (Chair)
Alan W Black (Co-Chair)
Louis-Philippe Morency
Amanda Stent (Bloomberg)

Copy of Thesis Document

For More Information, Please Contact: