I am a first year Ph.D. student in the Machine Learning Department at CMU, advised by Louis-Philippe Morency and Ruslan Salakhutdinov. My research is centered around multimodal machine learning, deep learning, semi-supervised learning, and unsupervised learning. From the applications perspective, I am interested in natural language processing, computer vision, and reinforcement learning. I received an M.S. in Machine Learning and a B.S. with University Honors in Computer Science from CMU, where I was extremely fortunate to have worked with Louis-Philippe Morency, Ruslan Salakhutdinov, Roni Rosenfeld, Barnabás Poczós and Tai Sing Lee.
News
12/2018: "Learning Factorized Multimodal Representations" to appear at ICLR 2019.
11/2018: 2 papers on deep learning accepted at AAAI 2019: multimodal cyclic translations and dynamic word representations.
11/2018: 4 papers to appear at NeurIPS 2018 workshops.
08/2018: 2 papers on multimodal NLP presented at EMNLP 2018 and ICMI 2018.
05/2018: I completed my Master's Data Analysis Project research and received the best presentation 1st runner-up award.
04/2018: 3 papers presented at ACL 2018 main conference and workshops: CMU-MOSEI dataset, low-rank fusion, and seq2seq2sentiment.
11/2017: 2 papers on neural networks accepted at AAAI 2018.
11/2017: Our paper on gated multimodal fusion won the honorable mention award at ICMI 2017.
Education
Carnegie Mellon University, Aug 2018 - Present
Doctor of Philosophy in Machine Learning
Advisors: Louis-Philippe Morency, Ruslan Salakhutdinov
GPA: 4.21/4.00
Carnegie Mellon University, Aug 2017 - May 2018
Master of Science in Machine Learning
Advisor: Louis-Philippe Morency
Thesis: Computational Modeling of Human Multimodal Language: The MOSEI Dataset and Interpretable Dynamic Fusion
GPA: 4.24/4.00
Carnegie Mellon University, Aug 2014 - May 2017
Bachelor of Science with University Honors in Computer Science
Minor in Neural Computation
GPA: 3.87/4.00
Publications
2019
Learning Factorized Multimodal Representations
Paul Pu Liang*, Yao-Hung Hubert Tsai*, Amir Zadeh, Louis-Philippe Morency, Ruslan Salakhutdinov (*equal contributions)
ICLR 2019 [paper][arXiv] [poster]
Found in Translation: Learning Robust Joint Representations by Cyclic Translations Between Modalities
Paul Pu Liang*, Hai Pham*, Thomas Manzini, Louis-Philippe Morency, Barnabás Póczos (*equal contributions)
AAAI 2019 [paper][supp][arXiv] [poster]
Words can Shift: Dynamically Adjusting Word Representations Using Nonverbal Behaviors
Yansen Wang, Ying Shen, Zhun Liu, Paul Pu Liang, Amir Zadeh, Louis-Philippe Morency
AAAI 2019 [paper][arXiv][code] [poster]
2018
Learning Robust Joint Representations for Multimodal Sentiment Analysis
Paul Pu Liang*, Hai Pham*, Thomas Manzini, Louis-Philippe Morency, Barnabás Póczos (*equal contributions)
NeurIPS 2018 Workshop on Interpretability and Robustness in Audio, Speech and Language (oral presentation) [paper][arXiv][slides]
Modeling Spatiotemporal Multimodal Language with Recurrent Multistage Fusion
Paul Pu Liang, Ziyin Liu, Amir Zadeh, Louis-Philippe Morency
NeurIPS 2018 Workshop on Modeling and Decision-making in the Spatiotemporal Domain (oral presentation) [paper][arXiv][slides][poster]
Learning Multimodal Representations with Factorized Deep Generative Models
Paul Pu Liang*, Yao-Hung Hubert Tsai*, Amir Zadeh, Louis-Philippe Morency, Ruslan Salakhutdinov (*equal contributions)
NeurIPS 2018 Workshop on Bayesian Deep Learning [paper][arXiv][poster]
Relational Attention Networks via Fully-Connected Conditional Random Fields
Ziyin Liu, Junxiang Chen, Paul Pu Liang, Masahito Ueda
NeurIPS 2018 Workshop on Bayesian Deep Learning [paper][poster]
Computational Modeling of Human Multimodal Language: The MOSEI Dataset and Interpretable Dynamic Fusion
Paul Pu Liang, Ruslan Salakhutdinov, Louis-Philippe Morency
Master's Thesis, CMU Machine Learning Data Analysis Project 2018 (best presentation runner-up) [paper][slides][poster]
Multimodal Language Analysis with Recurrent Multistage Fusion
Paul Pu Liang, Ziyin Liu, Amir Zadeh, Louis-Philippe Morency
EMNLP 2018 (oral presentation) [paper][supp][arXiv][slides]
An Empirical Evaluation of Sketched SVD and its Application to Leverage Score Ordering
Multimodal Language Analysis in the Wild: CMU-MOSEI Dataset and Interpretable Dynamic Fusion Graph
Amir Zadeh, Paul Pu Liang, Jonathan Vanbriesen, Soujanya Poria, Edmund Tong, Erik Cambria, Minghai Chen, Louis-Philippe Morency
ACL 2018 (oral presentation) [paper][supp][arXiv][slides][code]
Efficient Low-rank Multimodal Fusion with Modality-Specific Factors
Zhun Liu, Ying Shen, Varun Lakshminarasimhan, Paul Pu Liang, Amir Zadeh, Louis-Philippe Morency
ACL 2018 (oral presentation) [paper][arXiv][slides][code]
Proceedings of the First Grand Challenge and Workshop on Human Multimodal Language (Challenge-HML)
Seq2Seq2Sentiment: Multimodal Sequence to Sequence Models for Sentiment Analysis
Hai Pham, Thomas Manzini, Paul Pu Liang, Barnabás Póczos
ACL 2018 Grand Challenge and Workshop on Human Multimodal Language [paper][arXiv]
Multi-attention Recurrent Network for Human Communication Comprehension
Amir Zadeh, Paul Pu Liang, Soujanya Poria, Prateek Vij, Erik Cambria, Louis-Philippe Morency
AAAI 2018 (oral presentation) [paper][supp][arXiv][slides][code]
Memory Fusion Network for Multi-view Sequential Learning
Amir Zadeh, Paul Pu Liang, Navonil Mazumder, Soujanya Poria, Erik Cambria, Louis-Philippe Morency
AAAI 2018 (oral presentation) [paper][supp][arXiv][slides][code]
Robust Modulation Classification Under Uncertain Noise Conditions Using Recurrent Neural Networks
Shisheng Hu, Yiyang Pei, Paul Pu Liang, Ying-Chang Liang
GLOBECOM 2018 [paper]
Label-Assisted Transmission for Short Packet Communications: A Machine Learning Approach
Qianqian Zhang, Paul Pu Liang, Yu-Di Huang, Yiyang Pei, Ying-Chang Liang
IEEE TVT 2018 [paper]
A Machine Learning Approach to MIMO Communications
Yu-Di Huang, Paul Pu Liang, Qianqian Zhang, Ying-Chang Liang
ICC 2018 [paper]
2017
Multimodal Sentiment Analysis with Word-level Fusion and Reinforcement Learning
Paul Pu Liang*, Minghai Chen*, Sen Wang*, Tadas Baltrušaitis, Amir Zadeh, Louis-Philippe Morency (*equal contributions)
ICMI 2017 (oral presentation, honorable mention award) [paper][arXiv][slides]
Other Projects
On the Convergence, Generalization and Recovery Guarantees of Deep Neural Networks
Paul Pu Liang, Edgar Chen
10-702 Statistical Machine Learning Course Project, 2018 [paper]
Self-Supervised Dueling Networks for Deep Reinforcement Learning
Paul Pu Liang*, Xiang Xu* (*equal contributions)
10-703 Deep Reinforcement Learning Course Project, 2017 [paper][poster]
Machine Learning to Predict Neuronal Firing in a Multi-Neuron System
Paul Pu Liang, Stephen Tsou, Akshay Kulkarni
18-690 Neuroscience Course Project, 2017 [poster]
High Throughput Phenotyping with 3D CNNs in Ladder Networks
Hanqi Sun, Paul Pu Liang, Yue Niu
10-701 Machine learning (PhD) Course Project, 2016 [paper][poster]
Honors
CMU Graduate Research Fellowship, 2018-2023
CMU Machine Learning Department Data Analysis Project Best Presentation 1st Runner-up, 2018
NSF Graduate Research Fellowship Program Honorable Mention Award, 2018
ACM ICMI 2017 Honorable Mention Award, 2017
CMU School of Computer Science University Honors, 2017
CMU School of Computer Science Dean's List, 2015-2017
Harvard University Hacking Eating Tracking Hackathon Judge's Choice Award, 2015
Computational Modeling of Human Multimodal Language
RIKEN AIP Machine Learning Seminar, Tokyo, Japan, Jan 2019
RIKEN AIP Machine Learning Seminar, Kyoto, Japan, Dec 2018
University of Electronic Science and Technology of China, Chengdu, China, Aug 2018
ACL 2018, Melbourne, Australia, July 2018
Jilin University, Changchun, China, May 2018
CMU MLD Data Analysis Project Presentation, Pittsburgh, PA, USA, Apr 2018 [slides]
Learning Robust Joint Representations by Cyclic Translations Between Modalities
NeurIPS 2018 Workshop on Interpretability and Robustness, Montreal, Canada, Dec 2018 [slides]
Multimodal Language Analysis with Recurrent Multistage Fusion
NeurIPS 2018 Workshop on Spatiotemporal Modeling, Montreal, Canada, Dec 2018
EMNLP 2018, Brussels, Belgium, Nov 2018 [slides]
Advances in Multimodal Datasets
First Workshop on Human Multimodal Language at ACL 2018, Melbourne, Australia, July 2018 [slides]
Multi-attention Recurrent Network for Human Communication Comprehension
AAAI 2018, New Orleans, LA, USA, Feb 2018 [slides]
Multimodal Sentiment Analysis with Word-level Fusion and Reinforcement Learning
University of Electronic Science and Technology of China, Chengdu, China, Dec 2017 [slides]
Conference PC Member or Reviewer: ICLR, CVPR, ACL, EMNLP, NAACL, AAAI, ICMI, FG
Journal Reviewer: IEEE CIM, Elsevier Information Fusion
CMU Machine Learning Department PhD Admissions Committee
CMU Machine Learning Department Masters Admissions Committee
I have an Erdős number of 3 (Paul Erdős → Giuseppe Melfi → Erik Cambria → Paul Pu Liang).
This page has been accessed at least times since Feb 8, 2018.
@pliang279