Human face-to-face communication is a little like a dance: participants continuously adjust their behaviors based on their interlocutor’s speech, gestures and facial expressions during social interaction. The actual sophistication of human communication comes to the fore when we try to create computers that are able to understand and participate, however crudely, in this type of social interactions. My research takes up this challenge: building the computational foundations to enable computers with the abilities to analyze, recognize and predict subtle human communicative behaviors during social interactions. I formalize this new research endeavor with a Human Communication Dynamics framework, addressing four key computational challenges: behavioral dynamic, multimodal dynamic, interpersonal dynamic and societal dynamic.
Central to this research effort is the introduction of new probabilistic models able to learn the temporal and fine-grained dependencies across behaviors, modalities and interlocutors. In this talk, I will present a family of latent probabilistic models designed to automatically learn the hidden dynamics present in human verbal and nonverbal communication. These energy-based probabilistic models address core computational issues with human communication dynamics, including hierarchical and nonlinear feature representation, multi-view learning and potential over fitting on smaller training sets. My work takes advantage of recent advances in, and the potential for synergic interaction among, the fields of computational linguistics, machine learning and multimodal signal processing. Moreover, this research has clear applications in healthcare (depression, PTSD, suicide, autism), education (learning analytics), business (negotiation, interpersonal skills training) and social multimedia (opinion mining, social influence).
Louis-Philippe Morency is a Research Assistant Professor in the Department of Computer Science at the University of Southern California (USC) and leads the Multimodal Communication and Machine Learning Laboratory (MultiComp Lab) at the USC Institute for Creative Technologies. He received his Ph.D. and Master degrees from MIT Computer Science and Artificial Intelligence Laboratory. In 2008, Dr. Morency was selected as one of "AI's 10 to Watch" by IEEE Intelligent Systems. He has received 7 best paper awards in multiple ACM- and IEEE-sponsored conferences for his work on context-based gesture recognition, multimodal probabilistic fusion and computational models of human communication dynamics. For the past two years, Dr. Morency has been leading a DARPA-funded multi-institution effort called SimSensei which was recently named one of the year’s top ten most promising digital initiatives by the NetExplo Forum, in partnership with UNESCO.
Faculty Host: Eduard Hovy