Paul Pu Liang, CMU

Paul Pu Liang

Email: pliang(at)
Office: Gates and Hillman Center 8011
5000 Forbes Avenue, Pittsburgh, PA 15213
Multicomp Lab, Language Technologies Institute, School of Computer Science, Carnegie Mellon University

[CV] pliang279 @pliang279 @lpwinniethepu

I am a second-year Ph.D. student in the Machine Learning Department at Carnegie Mellon University, advised by Louis-Philippe Morency and Ruslan Salakhutdinov. I was also a research intern at Facebook AI Research, Nvidia AI, Google AI Research, and RIKEN Artificial Intelligence Project. My research is centered around multimodal machine learning, deep learning, and unsupervised learning. I apply these methods to problems in NLP, vision, speech, robotics, and healthcare. Previously, I received an M.S. in Machine Learning and a B.S. with University Honors in Computer Science from CMU.

[News] [Education] [Experience] [Publications] [Honors] [Teaching] [Students] [Talks] [Activities]





(* denotes joint first-authors)


  1. Anchor & Transform: Learning Sparse Representations of Discrete Objects
  2. Paul Pu Liang, Manzil Zaheer, Yuan Wang, Amr Ahmed
    preprint 2020
  3. Think Locally, Act Globally: Federated Learning with Local and Global Representations
  4. Paul Pu Liang*, Terrance Liu*, Liu Ziyin, Ruslan Salakhutdinov, Louis-Philippe Morency
    preprint 2020
    NeurIPS 2019 Workshop on Federated Learning (oral, distinguished student paper award)
    [arXiv] [code]
  5. Learning Not to Learn in the Presence of Noisy Labels
  6. Liu Ziyin, Blair Chen, Ru Wang, Paul Pu Liang, Ruslan Salakhutdinov, Louis-Philippe Morency, Masahito Ueda
    preprint 2020
  7. Diverse and Admissible Trajectory Prediction through Multimodal Context Understanding
  8. Seong Hyeon Park, Gyubok Lee, Manoj Bhat, Jimin Seo, Minseok Kang, Jonathan Francis, Ashwin Jadhav, Paul Pu Liang, Louis-Philippe Morency
    preprint 2020
  9. On Emergent Communication in Competitive Multi-Agent Teams
  10. Paul Pu Liang, Jeffrey Chen, Ruslan Salakhutdinov, Louis-Philippe Morency, Satwik Kottur
    AAMAS 2020 (oral)
    NeurIPS 2019 Workshop on Emergent Communication
    [arXiv] [code]


  1. Deep Gamblers: Learning to Abstain with Portfolio Theory
  2. Liu Ziyin, Zhikang Wang, Paul Pu Liang, Ruslan Salakhutdinov, Louis-Philippe Morency, Masahito Ueda
    NeurIPS 2019
    [arXiv] [code]
  3. Learning Representations from Imperfect Time Series Data via Tensor Rank Regularization
  4. Paul Pu Liang*, Zhun Liu*, Yao-Hung Hubert Tsai, Qibin Zhao, Ruslan Salakhutdinov, Louis-Philippe Morency
    ACL 2019
  5. Multimodal Transformer for Unaligned Multimodal Language Sequences
  6. Yao-Hung Hubert Tsai, Shaojie Bai, Paul Pu Liang, Zico Kolter, Louis-Philippe Morency, Ruslan Salakhutdinov
    ACL 2019
    [arXiv] [code]
  7. Social-IQ: A Question Answering Benchmark for Artificial Social Intelligence
  8. Amir Zadeh, Michael Chan, Paul Pu Liang, Edmund Tong, Louis-Philippe Morency
    CVPR 2019 (oral)
    [paper] [code]
  9. Strong and Simple Baselines for Multimodal Utterance Embeddings
  10. Paul Pu Liang*, Yao Chong Lim*, Yao-Hung Hubert Tsai, Ruslan Salakhutdinov, Louis-Philippe Morency
    NAACL 2019 (oral)
    [arXiv] [code] [slides]
  11. Learning Factorized Multimodal Representations
  12. Paul Pu Liang*, Yao-Hung Hubert Tsai*, Amir Zadeh, Louis-Philippe Morency, Ruslan Salakhutdinov
    ICLR 2019
    NeurIPS 2018 Workshop on Bayesian Deep Learning
    [arXiv] [code] [poster]
  13. Found in Translation: Learning Robust Joint Representations by Cyclic Translations Between Modalities
  14. Paul Pu Liang*, Hai Pham*, Thomas Manzini, Louis-Philippe Morency, Barnabás Póczos
    AAAI 2019
    NeurIPS 2018 Workshop on Interpretability and Robustness in Audio, Speech and Language (oral)
    [arXiv] [code] [slides] [poster]
  15. Words can Shift: Dynamically Adjusting Word Representations Using Nonverbal Behaviors
  16. Yansen Wang, Ying Shen, Zhun Liu, Paul Pu Liang, Amir Zadeh, Louis-Philippe Morency
    AAAI 2019
    [arXiv] [code] [slides] [poster]


  1. Computational Modeling of Human Multimodal Language: The MOSEI Dataset and Interpretable Dynamic Fusion
  2. Paul Pu Liang, Ruslan Salakhutdinov, Louis-Philippe Morency
    Master's Thesis, CMU Machine Learning Data Analysis Project 2018 (best presentation runner-up)
    [paper] [slides] [poster]
  3. Multimodal Language Analysis with Recurrent Multistage Fusion
  4. Paul Pu Liang, Ziyin Liu, Amir Zadeh, Louis-Philippe Morency
    EMNLP 2018 (oral)
    NeurIPS 2018 Workshop on Modeling and Decision-making in the Spatiotemporal Domain (oral)
    [arXiv] [slides] [poster]
  5. Multimodal Local-Global Ranking Fusion for Emotion Recognition
  6. Paul Pu Liang, Amir Zadeh, Louis-Philippe Morency
    ICMI 2018
    [arXiv] [poster]
  7. An Empirical Evaluation of Sketched SVD and its Application to Leverage Score Ordering
  8. Hui Han Chin, Paul Pu Liang
    ACML 2018
    [arXiv] [slides] [poster]
  9. Multimodal Language Analysis in the Wild: CMU-MOSEI Dataset and Interpretable Dynamic Fusion Graph
  10. Amir Zadeh, Paul Pu Liang, Jonathan Vanbriesen, Soujanya Poria, Edmund Tong, Erik Cambria, Minghai Chen, Louis-Philippe Morency
    ACL 2018 (oral)
    [arXiv] [code] [slides]
  11. Efficient Low-rank Multimodal Fusion with Modality-Specific Factors
  12. Zhun Liu, Ying Shen, Varun Lakshminarasimhan, Paul Pu Liang, Amir Zadeh, Louis-Philippe Morency
    ACL 2018 (oral)
    [arXiv] [code] [slides]
  13. Proceedings of the First Grand Challenge and Workshop on Human Multimodal Language (Challenge-HML)
  14. Amir Zadeh, Paul Pu Liang, Louis-Philippe Morency, Soujanya Poria, Erik Cambria, Stefan Scherer
    ACL 2018 Workshop Proceedings
    [proceedings] [website] [introduction] [datasets] [results]
  15. Multi-attention Recurrent Network for Human Communication Comprehension
  16. Amir Zadeh, Paul Pu Liang, Soujanya Poria, Prateek Vij, Erik Cambria, Louis-Philippe Morency
    AAAI 2018 (oral)
    [arXiv] [code] [slides]
  17. Memory Fusion Network for Multi-view Sequential Learning
  18. Amir Zadeh, Paul Pu Liang, Navonil Mazumder, Soujanya Poria, Erik Cambria, Louis-Philippe Morency
    AAAI 2018 (oral)
    [arXiv] [code] [slides]


  1. Multimodal Sentiment Analysis with Word-level Fusion and Reinforcement Learning
  2. Paul Pu Liang*, Minghai Chen*, Sen Wang*, Tadas Baltrušaitis, Amir Zadeh, Louis-Philippe Morency
    ICMI 2017 (oral, honorable mention award)
    [arXiv] [code] [slides]



Student Advising

Academic Talks

Professional Activities

I have an Erdős number of 3 (Paul Erdős → Giuseppe Melfi → Erik Cambria → Paul Pu Liang).
This page has been accessed at least several times since Feb 8, 2018.