Louis-Philippe Morency

Assistant Professor, Language Technology Institute,

School of Computer Science, Carnegie Mellon University

Director, MultiComp Lab (website currently hosted at USC)

Gates-Hillman Center (GHC) Office 5411,

5000 Forbes Avenue, Pittsburgh, PA 15213

Email: morency@cs.cmu.edu

Phone: (412) 268-5508


I am tenure-track Faculty at CMU Language Technology Institute where I lead the Multimodal Communication and Machine Learning Laboratory (MultiComp Lab). I was previously Research Faculty at USC Computer Science Department. I received my Ph.D. in Computer Science from MIT Computer Science and Artificial Intelligence Laboratory.

My research focuses on building the computational foundations to enable computers with the abilities to analyze, recognize and predict subtle human communicative behaviors during social interactions. I have formalized this new research endeavor with the Human Communication Dynamics framework, addressing four key computational challenges: behavioral dynamic, multimodal dynamic, interpersonal dynamic and societal dynamic. This multi-disciplinary research topic overlaps the fields of multimodal interaction, social psychology, computer vision, machine learning and artificial intelligence, and has many applications in areas as diverse as medicine, robotics and education.

Human Communication Dynamics

·       Analyze, recognize and predict subtle human communicative behaviors during social interactions.

Multimodal Machine Learning

·       Probabilistic modeling of acoustic, visual and verbal modalities

·       Learning the temporal contingency between modalities

Health Behavior Informatics

·       Technologies to support clinical practice during diagnosis and treatment of mental health disorders


-          Congratulations Sunghyun Park for his successful PhD thesis defense!!

-          Best paper award, ACM International Conference on Multimodal Interaction, Seattle, WA, 2015

-          NetExplo 2014 Award, in partnership with UNESCO, as one of the year’s top ten most promising digital initiatives

-          Program Chair, International Conference on Multimodal Interaction (ICMI 2014), Istanbul, Turkey, November 12-16 2014

-          Best paper award, 10th IEEE International Conference Automatic Face and Gesture Recognition (FG 2013), Shanghai, China April 22-26, 2013

-          Best paper award nomination, The fifth biannual Conference on Affective Computing and Intelligent Interaction (ACII 2013), Geneva, Switzerland, August 22-24, 2013

-          Best paper award nomination, 14th annual SIGdial Meeting on Discourse and Dialogue (SigDial 2013), Metz, France, September 2-5, 2013

-          General Chair, International Conference on Multimodal Interaction (ICMI 2012), Santa Monica, CA, October 22-26, 2012

Selected Publications (see full list in my CV, MultiComp Lab or Google Scholar)

§  M. Chatterjee, S. Park, S. Scherer and L.-P. Morency. Combining Two Perspectives on Classifying Multimodal Data for Recognizing Speaker Traits. In Proceedings of the ACM International Conference on Multimodal Interaction (ICMI), 2015 *** Best Paper Award ***

§  K. Bousmalis, S. Zafeiriou, L.-P. Morency, M. Pantic and Z. Ghahramani, Variational Infinite Hidden Conditional Random Fields, IEEE Transaction on Pattern Analysis and Machine Intelligence (TPAMI), Volume 37, Issue 9, September 2015, Pages 1917-1929

§  T. Wortwein, M. Chollet, A. Shapiro, S. Scherer, and L.-P. Morency. Exploring Feedback Learning Strategies to Improve Public Speaking: An Interactive Virtual Audience Framework. ACM International Joint Conference on Pervasive and Ubiquitous Computing (Ubicomp), 2015

§  T. Baltrusaitis, P. Robinson and L.-P. Morency, Continuous Conditional Neural Fields for Structured Regression, ECCV 2014

§  C. Miller, F. Quek and L.-P. Morency, Search Strategies for Pattern Identification in Multimodal Data: Three Case Studies. ICMR 2014

§  J. Gratch, G. Lucas, A. King, L.-P. Morency, It’s only a computer: the impact of human-agent interaction in clinical interviews, AAMAS 2014

§  M. Chatterjee, G. Stratou, S. Scherer and L.-P. Morency, Context-based Signal Descriptors of Heart-Rate Variability for Anxiety Assessment, ICASSP 2014

§  S. Park, P. Shoemark, and L.-P. Morency.“Toward Crowdsourcing Micro-Level Behavior Annotations: The Challenges of Interface, Training, and Generalization.” IUI 2014, ***Best Paper Award finalist ***

§  Y Song, L.-P. Morency and R. Davis. Action Recognition by Hierarchical Sequence Summarization. CVPR 2013

§  V. P. Rosas, R. Mihalcea and L.-P. Morency. Utterance-Level Multimodal Sentiment Analysis. ACL 2013

§  S. Scherer, G. Stratou, J. Boberg, J. Gratch, A. Rizzo and L.-P. Morency. Automatic Behavior Descriptors for Psychological Disorder Analysis. FG 2013, *** Best paper award ***

§  K. Bousmalis, S. Zafeiriou, L.-P. Morency, M. Pantic and Z. Ghahramani. Variational Hidden Conditional Random Fields with Coupled Dirichlet Process Mixtures. ECML 2013

§  S. Scherer, J. Pestian, and L.-P. Morency. Investigating the Speech Characteristics of Suicidal Adolescents. ICASSP 2013

§  D. DeVault, K. Georgila, R. Artstein, F. Morbini, D. Traum, S. Scherer, A. Rizzo, L.-P. Morency. Verbal Indicators of Psychological Distress in Interactive Dialogue with a Virtual Human. In SigDial 2013, ***Best Paper Award finalist ***

§  S. Park, S. Scherer, J. Gratch, P. Carnevale, and L.-P. Morency. Mutual Behaviors during Dyadic Negotiation: Automatic Prediction of Respondent Reactions. ACII 2013 , ***Best Paper Award finalist ***

Graduate Students Advising

Amir Ali Bagherzade, Ph.D. program (CMU)

Chaitanya Ahuja, Ph.D. program (CMU)

Volkan Cirik, M.S. program (CMU)

Hongliang Yu, M.S. program (CMU)

Sayan Ghosh, Ph.D. program (USC, with Prof. Stefan Scherer)

KangGeon Kim, Ph.D. program (USC, with Prof. Gerard Medioni)


       CMU-11777: Advanced Multimodal Machine Learning, Spring 2016

Fundamental mathematical concepts related to multimodal machine learning including multimodal alignment and fusion, heterogeneous representation learning and multi-stream temporal modeling.

       CMU-11776: Human Communication and Multimodal Machine Learning, Fall 2015, 2016

Recent advances in machine learning, pattern recognition and signal processing to analyze, recognize and predict human communication behaviors during social interactions.

       USC-CS599: Human Communication and Machine Learning, Fall 2010, 2012, 2014

Selected Press Coverage