Paul Pu Liang, CMU

Paul Pu Liang

Email: pliang(at)cs.cmu.edu
Office: Gates and Hillman Center 8011
5000 Forbes Avenue, Pittsburgh, PA 15213
Multicomp Lab, Language Technologies Institute, School of Computer Science, Carnegie Mellon University
[CV]

@lpwinniethepu

I am a first year Ph.D. student in the Machine Learning Department at CMU, advised by Louis-Philippe Morency and Ruslan Salakhutdinov. My research is centered around multimodal machine learning, deep learning, semi-supervised learning, and unsupervised learning. I apply these methods to problems in natural language processing, computer vision, and reinforcement learning. I received an M.S. in Machine Learning and a B.S. with University Honors in Computer Science from CMU, where I was extremely fortunate to have worked with Louis-Philippe Morency, Ruslan Salakhutdinov, Roni Rosenfeld, Barnabás Poczós and Tai Sing Lee. I have also spent time as a research intern at Google Research and RIKEN Artificial Intelligence Project.

News

02/2019: 2 papers accepted at NAACL and CVPR 2019, preprints and a new QA dataset on comprehending social interactions are coming soon!
01/2019: I will be a TA for 10-708 Probabilistic Graphical Models in Spring 2019, with new content on deep generative models, RL, and probabilistic programming!
12/2018: "Learning Factorized Multimodal Representations" to appear at ICLR 2019.
11/2018: 2 papers on deep learning accepted at AAAI 2019: multimodal cyclic translations and dynamic word representations.
11/2018: I will be attending NeurIPS 2018 to present some workshop papers on disentangled and robust multimodal representation learning.
08/2018: 2 papers on multimodal NLP [1,2] presented at EMNLP 2018 and ICMI 2018.
05/2018: I completed my Master's Data Analysis Project research and received the best presentation 1st runner-up award.
04/2018: 3 papers presented at ACL 2018 main conference and workshops: CMU-MOSEI dataset, low-rank fusion, and seq2seq2sentiment.
01/2018: I will be co-organizing the First Grand Challenge and Workshop on Human Multimodal Language at ACL 2018.
11/2017: 2 papers on neural networks [1,2] accepted at AAAI 2018.
11/2017: Our paper on gated multimodal fusion won the honorable mention award at ICMI 2017!

Education

Carnegie Mellon University, Aug 2018 - Present

Carnegie Mellon University, Aug 2017 - May 2018

Carnegie Mellon University, Aug 2014 - May 2017

Publications

2019

A Strong and Simple Baseline for Multimodal Utterance Embeddings

oral presentation

Social-IQ: A Question Answering Benchmark for Open-ended Social Intelligence

oral presentation

Learning Factorized Multimodal Representations

Found in Translation: Learning Robust Joint Representations by Cyclic Translations Between Modalities

Words can Shift: Dynamically Adjusting Word Representations Using Nonverbal Behaviors

2018

Computational Modeling of Human Multimodal Language: The MOSEI Dataset and Interpretable Dynamic Fusion

best presentation runner-up

Multimodal Language Analysis with Recurrent Multistage Fusion

oral presentation

An Empirical Evaluation of Sketched SVD and its Application to Leverage Score Ordering

Multimodal Local-Global Ranking Fusion for Emotion Recognition

ICMI 2018

Multimodal Language Analysis in the Wild: CMU-MOSEI Dataset and Interpretable Dynamic Fusion Graph

oral presentation

Efficient Low-rank Multimodal Fusion with Modality-Specific Factors

oral presentation

Proceedings of the First Grand Challenge and Workshop on Human Multimodal Language (Challenge-HML)

Multi-attention Recurrent Network for Human Communication Comprehension

oral presentation

Memory Fusion Network for Multi-view Sequential Learning

oral presentation

2017

Multimodal Sentiment Analysis with Word-level Fusion and Reinforcement Learning

oral presentation

honorable mention award

Honors

CMU Graduate Research Fellowship, 2018-2023
CMU Machine Learning Department Data Analysis Project Best Presentation 1st Runner-up, 2018
NSF Graduate Research Fellowship Program Honorable Mention Award, 2018
ACM ICMI 2017 Honorable Mention Award, 2017
CMU School of Computer Science University Honors, 2017
CMU School of Computer Science Dean's List, 2015-2017
Harvard University Hacking Eating Tracking Hackathon Judge's Choice Award, 2015

Teaching

TA: 10-708 Probabilistic Graphical Models, Spring 2019, CMU. Instructor: Eric Xing
TA: 10-715 Advanced Introduction to Machine Learning, Fall 2018, CMU. Instructor: Maria-Florina Balcan
TA: 10-601 Introduction to Machine Learning, Fall 2016, CMU. Instructor: Roni Rosenfeld
TA: 15-213/18-213/15-513 Introduction to Computer Systems, Summer 2016, CMU. Instructor: Brian Railing

Academic Talks

Computational Modeling of Human Multimodal Language

Learning Robust Joint Representations by Cyclic Translations Between Modalities

Multimodal Language Analysis with Recurrent Multistage Fusion

Advances in Multimodal Datasets

Multi-attention Recurrent Network for Human Communication Comprehension

Multimodal Sentiment Analysis with Word-level Fusion and Reinforcement Learning

Professional Activities

Workflow Co-Chair: ICML 2019
Session Co-Chair: AAAI 2019 (NLP session)
General Co-Chair: ACL 2018 First Grand Challenge and Workshop on Human Multimodal Language
Conference PC Member or Reviewer: ICLR, ACL, EMNLP, NAACL, CVPR, ICCV, AAAI, ICMI, FG
Journal Reviewer: IEEE Transactions on Affective Computing, IEEE Computational Intelligence Magazine, Elsevier Information Fusion, Elsevier Computer Speech and Language
CMU Machine Learning Department PhD Admissions Committee
CMU Machine Learning Department Masters Admissions Committee

I have an Erdős number of 3 (Paul Erdős → Giuseppe Melfi → Erik Cambria → Paul Pu Liang).
This page has been accessed at least

several

times since Feb 8, 2018.