Paul Pu Liang, CMU

Paul Pu Liang

Email: pliang(at)cs.cmu.edu
Office: Gates and Hillman Center 8011
5000 Forbes Avenue, Pittsburgh, PA 15213
Multicomp Lab, Language Technologies Institute, School of Computer Science, Carnegie Mellon University

[CV] @pliang279 @pliang279 @lpwinniethepu
I am a second-year Ph.D. student in the Machine Learning Department at Carnegie Mellon University, advised by Louis-Philippe Morency and Ruslan Salakhutdinov. I was also a research intern at Facebook AI Research, Nvidia AI, Google AI Research, and RIKEN Artificial Intelligence Project. My research is centered around multimodal machine learning, deep learning, and unsupervised learning. I apply these methods to problems in NLP, vision, speech, robotics, and healthcare. Previously, I received an M.S. in Machine Learning and a B.S. with University Honors in Computer Science from CMU.

[News] [Education] [Experience] [Publications] [Honors] [Teaching] [Students] [Talks] [Activities]

News

Education

Experience

Publications

(* denotes joint first-authors)

2020

  1. Anchor & Transform: Learning Sparse Representations of Discrete Objects
    2. Paul Pu Liang, Manzil Zaheer, Yuan Wang, Amr Ahmed
    preprint
    [arXiv]
  2. Cross-Modal Generalization via Meta Alignment
    3. Paul Pu Liang, Peter Wu, Liu Ziyin, Louis-Philippe Morency, Ruslan Salakhutdinov
    preprint
    [arXiv]
  3. Think Locally, Act Globally: Federated Learning with Local and Global Representations
    4. Paul Pu Liang*, Terrance Liu*, Liu Ziyin, Nicholas B. Allen, Randy P. Auerbach, David Brent, Ruslan Salakhutdinov, Louis-Philippe Morency
    preprint
    NeurIPS 2019 Workshop on Federated Learning (oral, distinguished student paper award)
    [arXiv] [code]
  4. Learning Not to Learn in the Presence of Noisy Labels
    5. Liu Ziyin, Blair Chen, Ru Wang, Paul Pu Liang, Ruslan Salakhutdinov, Louis-Philippe Morency, Masahito Ueda
    preprint
    [arXiv]
  5. Variational Auto-Decoder
    6. Amir Zadeh, Yao-Chong Lim, Paul Pu Liang, Louis-Philippe Morency
    preprint
    [arXiv] [code]
  6. Diverse and Admissible Trajectory Prediction through Multimodal Context Understanding
    7. Seong Hyeon Park, Gyubok Lee, Manoj Bhat, Jimin Seo, Minseok Kang, Jonathan Francis, Ashwin R. Jadhav, Paul Pu Liang, Louis-Philippe Morency
    ECCV 2020
    CVPR 2020 Argoverse competition (honorable mention award)
    [arXiv]
  7. Towards Debiasing Sentence Representations
    8. Paul Pu Liang, Irene Li, Emily Zheng, Yao Chong Lim, Ruslan Salakhutdinov, Louis-Philippe Morency
    ACL 2020
    [arXiv] [code]
  8. Proceedings of the Second Grand Challenge and Workshop on Human Multimodal Language (Challenge-HML)
    9. Amir Zadeh, Louis-Philippe Morency, Paul Pu Liang, Soujanya Poria
    ACL 2020 Workshop Proceedings
    [proceedings] [website]
  9. On Emergent Communication in Competitive Multi-Agent Teams
    10. Paul Pu Liang, Jeffrey Chen, Ruslan Salakhutdinov, Louis-Philippe Morency, Satwik Kottur
    AAMAS 2020 (oral)
    NeurIPS 2019 Workshop on Emergent Communication
    [arXiv] [code] [slides]
  10. Empirical and Theoretical Studies of Multimodal Co-learning
    11. Amir Zadeh, Paul Pu Liang, Louis-Philippe Morency
    Elsevier Information Fusion 2020
    [arXiv]

2019

  1. Deep Gamblers: Learning to Abstain with Portfolio Theory
    2. Liu Ziyin, Zhikang Wang, Paul Pu Liang, Ruslan Salakhutdinov, Louis-Philippe Morency, Masahito Ueda
    NeurIPS 2019
    [arXiv] [code] [poster]
  2. Learning Representations from Imperfect Time Series Data via Tensor Rank Regularization
    3. Paul Pu Liang*, Zhun Liu*, Yao-Hung Hubert Tsai, Qibin Zhao, Ruslan Salakhutdinov, Louis-Philippe Morency
    ACL 2019
    [arXiv] [poster]
  3. Multimodal Transformer for Unaligned Multimodal Language Sequences
    4. Yao-Hung Hubert Tsai, Shaojie Bai, Paul Pu Liang, Zico Kolter, Louis-Philippe Morency, Ruslan Salakhutdinov
    ACL 2019
    [arXiv] [code]
  4. Social-IQ: A Question Answering Benchmark for Artificial Social Intelligence
    5. Amir Zadeh, Michael Chan, Paul Pu Liang, Edmund Tong, Louis-Philippe Morency
    CVPR 2019 (oral)
    [paper] [code] [poster]
  5. Strong and Simple Baselines for Multimodal Utterance Embeddings
    6. Paul Pu Liang*, Yao Chong Lim*, Yao-Hung Hubert Tsai, Ruslan Salakhutdinov, Louis-Philippe Morency
    NAACL 2019 (oral)
    [arXiv] [code] [slides]
  6. Learning Factorized Multimodal Representations
    7. Paul Pu Liang*, Yao-Hung Hubert Tsai*, Amir Zadeh, Louis-Philippe Morency, Ruslan Salakhutdinov
    ICLR 2019
    NeurIPS 2018 Workshop on Bayesian Deep Learning
    [arXiv] [code] [poster]
  7. Found in Translation: Learning Robust Joint Representations by Cyclic Translations Between Modalities
    8. Paul Pu Liang*, Hai Pham*, Thomas Manzini, Louis-Philippe Morency, Barnabás Póczos
    AAAI 2019
    NeurIPS 2018 Workshop on Interpretability and Robustness in Audio, Speech and Language (oral)
    [arXiv] [code] [slides] [poster]
  8. Words can Shift: Dynamically Adjusting Word Representations Using Nonverbal Behaviors
    9. Yansen Wang, Ying Shen, Zhun Liu, Paul Pu Liang, Amir Zadeh, Louis-Philippe Morency
    AAAI 2019
    [arXiv] [code] [slides] [poster]

2018

  1. Computational Modeling of Human Multimodal Language: The MOSEI Dataset and Interpretable Dynamic Fusion
    2. Paul Pu Liang, Ruslan Salakhutdinov, Louis-Philippe Morency
    Master's Thesis, CMU Machine Learning Data Analysis Project 2018 (first runner-up award)
    [paper] [slides] [poster]
  2. Multimodal Language Analysis with Recurrent Multistage Fusion
    3. Paul Pu Liang, Ziyin Liu, Amir Zadeh, Louis-Philippe Morency
    EMNLP 2018 (oral)
    NeurIPS 2018 Workshop on Modeling and Decision-making in the Spatiotemporal Domain (oral)
    [arXiv] [slides] [poster]
  3. Multimodal Local-Global Ranking Fusion for Emotion Recognition
    4. Paul Pu Liang, Amir Zadeh, Louis-Philippe Morency
    ICMI 2018
    [arXiv] [poster]
  4. An Empirical Evaluation of Sketched SVD and its Application to Leverage Score Ordering
    5. Hui Han Chin, Paul Pu Liang
    ACML 2018
    [arXiv] [slides] [poster]
  5. Multimodal Language Analysis in the Wild: CMU-MOSEI Dataset and Interpretable Dynamic Fusion Graph
    6. Amir Zadeh, Paul Pu Liang, Jonathan Vanbriesen, Soujanya Poria, Edmund Tong, Erik Cambria, Minghai Chen, Louis-Philippe Morency
    ACL 2018 (oral)
    [arXiv] [code] [slides]
  6. Efficient Low-rank Multimodal Fusion with Modality-Specific Factors
    7. Zhun Liu, Ying Shen, Varun Lakshminarasimhan, Paul Pu Liang, Amir Zadeh, Louis-Philippe Morency
    ACL 2018 (oral)
    [arXiv] [code] [slides]
  7. Proceedings of the First Grand Challenge and Workshop on Human Multimodal Language (Challenge-HML)
    8. Amir Zadeh, Paul Pu Liang, Louis-Philippe Morency, Soujanya Poria, Erik Cambria, Stefan Scherer
    ACL 2018 Workshop Proceedings
    [proceedings] [website] [introduction] [datasets] [results]
  8. Multi-attention Recurrent Network for Human Communication Comprehension
    9. Amir Zadeh, Paul Pu Liang, Soujanya Poria, Prateek Vij, Erik Cambria, Louis-Philippe Morency
    AAAI 2018 (oral)
    [arXiv] [code] [slides]
  9. Memory Fusion Network for Multi-view Sequential Learning
    10. Amir Zadeh, Paul Pu Liang, Navonil Mazumder, Soujanya Poria, Erik Cambria, Louis-Philippe Morency
    AAAI 2018 (oral)
    [arXiv] [code] [slides]

2017

  1. Multimodal Sentiment Analysis with Word-level Fusion and Reinforcement Learning
    2. Paul Pu Liang*, Minghai Chen*, Sen Wang*, Tadas Baltrušaitis, Amir Zadeh, Louis-Philippe Morency
    ICMI 2017 (oral, honorable mention award)
    [arXiv] [code] [slides]

Honors

Teaching

Student Advising

Academic Talks

Professional Activities


I have an Erdős number of 3 (Paul Erdős → Giuseppe Melfi → Erik Cambria → Paul Pu Liang).
This page has been accessed at least several times since Feb 8, 2018.