Paul Liang, CMU

Paul Pu Liang

Email: pliang(at)cs.cmu.edu
Office: Gates and Hillman Center 8011
5000 Forbes Avenue, Pittsburgh, PA 15213
Machine Learning Department and Language Technologies Institute, School of Computer Science, Carnegie Mellon University

[CV]

[google scholar]

[linkedin]

[github]

[twitter]

[instagram]

[News] [Education] [Publications] [Honors] [Teaching] [Talks] [Activities]

I am a final-year Ph.D. student in the Machine Learning Department at Carnegie Mellon University, advised by Louis-Philippe Morency and Ruslan Salakhutdinov. I also collaborate with Manuel Blum, Lenore Blum, Jack Hessel, and Yejin Choi at Berkeley and UW/AI2.

I am on the academic job market this year. [CV] [research statement] [teaching statement] [DEI statement]

My research lies in the foundations of multimodal machine learning with applications in socially intelligent AI, natural language processing, healthcare, and education. As steps towards this goal, I work on:

Foundations of modality heterogeneity, connections, and interactions [foundations, interactions, disagreement, FactorCL, MultiViz, factorized, translation].

Representation learning over multisensory and temporal data [HighMMT, MultiBench, T2FN, RMFN, MulT].

Multimodal applications in socially intelligent AI, NLP, healthcare, and education [CMU-MOSEI, Social-IQ, mental health, pathology, education].

Real-world fair, robust, interpretable, and efficient representation learning [fairness in LMs, fairness in BERT, federated learning, efficiency].

My research is generously supported by a Siebel Scholars Award, Waibel Presidential Fellowship, Facebook PhD Fellowship, and Center for Machine Learning and Health Fellowship, and has been recognized by 3 best paper/honorable mention awards at NeurIPS workshops and ICMI 2017. I love teaching and advising, and was honored to receive the Alan J. Perlis Graduate Student Teaching Award for co-instructing courses (CMU 11-877, CMU 11-866, CMU 11-777), and organizing workshops (ICCV, NAACL, NeurIPS, ACL) and tutorials (ICML, CVPR, NAACL) on multimodal machine learning. Previously, I received an M.S. in Machine Learning and a B.S. with University Honors in Computer Science and Neural Computation from CMU, where I am grateful for the mentorship of Louis-Philippe Morency, Ruslan Salakhutdinov, Tai Sing Lee, Roni Rosenfeld, and Ryan Tibshirani. I have also been fortunate to spend time at DeepMind, Facebook AI Research, Nvidia AI, Google Research, and RIKEN Artificial Intelligence Project.

Research opportunities: I am happy to collaborate and answer questions about my research and CMU academic programs. If you are interested, please send me an email. I especially encourage students from underrepresented groups to reach out.

News

2023: Excited to release some recent work formalizing and quantifying multimodal interactions from statistical (NeurIPS 2023) and human (ICMI 2023) perspectives, with applications in visualizing and interpreting multimodal models (ICLR 2023), contrastive learning of unique information (NeurIPS 2023), and guarantees for multimodal semi-supervised learning (arXiv 2023).
2023: Co-teaching 11-777 Multimodal Machine Learning, Fall 2023, course content will be updated on the website.
2023: Tutorials on multimodal machine learning at ICML 2023, ICMI 2023, CVPR 2022 and NAACL 2022, teaching at CIFAR DLRL summer school and African Masters of Machine Intelligence (day1, day2, day3, day4): check out our survey paper, slides, and videos.
2023: Nothing has excited me more than collaborating with and advising great students. I've learned so much from them and I'm excited to watch them embark on their new research agendas - follow their work for more exciting new ideas! Yun Cheng -> PhD at Princeton, Rulin Shao -> PhD at UW, Xiang Fan -> PhD at UW, Jivat Neet -> PhD at Berkeley, Yiwei Lyu -> PhD at Michigan, Yuxin Xiao -> PhD at MIT, Peter Wu -> PhD at Berkeley, Dong Won Lee -> PhD at MIT, Terrance Liu -> PhD at CMU.
2023: LP and I are teaching 2 new graduate seminar courses: 11-866 Artificial Social Intelligence and 11-877 Advanced Topics in Multimodal Machine Learning.
2022: Check out course content for 11-777 Multimodal Machine Learning, Fall 2022, where LP and I have completely revamped the course content. Also check out fully recorded lecture videos and course content for 11-777 in Fall 2020.
2022: Are you working on multimodal tasks and can't decide on a model? Check out HighMMT (TMLR 2022), our attempt at a single multimodal model that can predict sentiment, emotion, humor, disease, robot pose, and more, as well as MultiBench (NeurIPS 2021) and MultiZoo (JMLR 2022), a large-scale benchmark for multimodal learning spanning 15 datasets, 10 modalities, 20 prediction tasks, and 6 research areas.
2021: Extremely honored to have received a Facebook PhD Fellowship and a Center for Machine Learning and Health Fellowship to support my research in socially intelligent AI! For students applying for graduate fellowships, I uploaded my statement from the 2020 application.
2020: Check out the CMU Machine Learning Blog - new research and educational content every few weeks on ML research going on at CMU!