Jielin Qiu
jielinq [at] cs (dot) cmu (dot) edu

I am a PhD student in the Computer Science Department at School of Computer Science, Carnegie Mellon University, advised by Prof. Ding Zhao. I also work closely with Prof. Bo Li and Prof. Douglas Weber. My research interests lie in Multimodal Machine Learning, I am interested in designing scalable inference and learning algorithms to connect language, perception, and actions for robust multimodal interaction. My current research lies in the foundations of multimodal learning with applications in multimedia, computer vision, natural language processing, cognition, and healthcare.

Before coming to CMU, I received my B.Eng. from Shanghai Jiao Tong University, advised by Prof. Bao-Liang Lu.

Email  /  Google Scholar  /  LinkedIn

profile photo
Update
  • If you're an undergraduate or master student interested in RA/summer internship, please feel free to contact me. Our lab is looking for several students for research projects.
News
  • [2022-12] NeurIPS 2022 virtual deep-dive session chairs.
  • [2022-10] One paper accepted by WACV 2023.
  • [2022-10] One paper accepted by NeurIPS 2022 Workshop on Distribution Shifts.
  • [2022-10] Top Reviewers in NeurIPS 2022.
  • [2022-06] One paper accepted by MLHC 2022.
  • [2022-05] Start a research internship at AWS AI.
  • [2022-05] One paper accepted by ICML 2022 workshop on Principles of Distribution Shift.
  • [2022-04] One paper accepted by ICLR 2022 Workshop on Socially Responsible Machine Learning.
  • [2021-09] Receive a gift funding from Adobe. Thanks, Adobe!
  • [2021-08] TA 16-824 Visual Learning and Recognition by Prof. Jun-Yan Zhu. Check our course here: 16-824 Fall2021.
  • [2021-05] Start a research internship at Adobe research.
  • [2021-01] TA 11-777 MultiModal Machine Learning by Prof. Yonatan Bisk. Check our course here: 11-777 Spring2021.
Selected Publications

* marked as equal contribution

Are Multimodal Models Robust to Image and Text Perturbations?
Jielin Qiu, Yi Zhu, Xingjian Shi, Florian Wenzel, Zhiqiang Tang, Ding Zhao, Bo Li, Mu Li
Under Review

Benchmarking Robustness under Distribution Shift of Multimodal Image-Text Models
Jielin Qiu, Yi Zhu, Xingjian Shi, Zhiqiang Tang, Ding Zhao, Bo Li, Mu Li
NeurIPS 2022 Workshop on Distribution Shifts

Semantics-Consistent Cross-domain Summarization via Optimal Transport Alignment
Jielin Qiu, Jiacheng Zhu, Mengdi Xu, Franck Dernoncourt, Trung Bui, Zhaowen Wang, Bo Li, Ding Zhao, Hailin Jin
Under Review / arXiv

LiveSeg: Unsupervised Multimodal Temporal Segmentation of Long Livestream Videos
Jielin Qiu, Franck Dernoncourt, Trung Bui, Zhaowen Wang, Ding Zhao, Hailin Jin
WACV 2023 / arXiv

An Empirical Exploration of Cross-domain Alignment between Language and Electroencephalogram
William Han*, Jielin Qiu*, Jiacheng Zhu, Mengdi Xu, Douglas Weber, Bo Li, Ding Zhao
Under Review / arXiv

MHMS: Multimodal Hierarchical Multimedia Summarization
Jielin Qiu, Jiacheng Zhu, Mengdi Xu, Franck Dernoncourt, Trung Bui, Zhaowen Wang, Bo Li, Ding Zhao, Hailin Jin
Under Review / arXiv

Unsupervised Multimodal Temporal Segmentation of Long Livestream Videos
Jielin Qiu, Franck Dernoncourt, Trung Bui, Zhaowen Wang, Ding Zhao, Hailin Jin
Under Review

Group Distributionally Robust Reinforcement Learning with Hierarchical Latent Variables
Mengdi Xu, Peide Huang, Visak Kumar, Jielin Qiu, Chao Fang, Kuan-Hui Lee, Xuewei Qi, Henry Lam, Bo Li, Ding Zhao
Under Review / paper

GeoECG: Data Augmentation via Wasserstein Geodesic Perturbation for Robust Electrocardiogram Prediction
Jiacheng Zhu*, Jielin Qiu*, Zhuolin Yang, Douglas Weber, Michael Rosenberg, Emerson Liu, Bo Li, Ding Zhao
MLHC 2022 / arXiv

Data Augmentation via Wasserstein Geodesic Perturbation for Robust Electrocardiogram Prediction
Jiacheng Zhu*, Jielin Qiu*, Zhuolin Yang, Michael Rosenberg, Emerson Liu, Bo Li, Ding Zhao
ICLR 2022 Workshop on Socially Responsible Machine Learning (SRML) / Paper

Optimal Transport based Data Augmentation for Heart Disease Diagnosis and Prediction
Jielin Qiu*, Jiacheng Zhu*, Michael Rosenberg, Emerson Liu, Ding Zhao
preprint / arXiv

Comparing Recognition Performance and Robustness of Multimodal Deep Learning Models for Multimodal Emotion Recognition
Wei Liu, Jielin Qiu, Wei-Long Zheng, Bao-Liang Lu
IEEE Transactions on Cognitive and Developmental Systems 2021 / Paper / arXiv

Visual Sequence Learning in Hierarchical Prediction Networks and Primate Visual Cortex
Jielin Qiu, Ge Huang, Tai Sing Lee
NeurIPS 2019 / Paper

Investigating Sex Differences in Classification of Five Emotions from EEG and Eye Movement Signals
Lan-Qing Bao, Jielin Qiu, Hao Tang, Wei-Long Zheng, Bao-Liang Lu
EMBC 2019 / Paper

Approximation Gradient Error Variance Reduced Optimization
Weiye Zhao, Yang Liu, Xiaoming Zhao, Jielin Qiu, Jian Peng
AAAI 2019 / Paper

Multi-view Emotion Recognition Using Deep Canonical Correlation Analysis
Jielin Qiu, Wei Liu, Bao-Liang Lu
ICONIP 2018 / Paper

Services
  • Reviewer: CVPR 2023, EACL 2023, ICASSP 2023, WACV 2023, AISTATS 2023, NeurIPS 2022, CVPR 2022, ICML 2022, ECCV2022, EMNLP2022, ACM MM 2022, MLHC 2022, CHIL 2022, ICML 2021.
  • PC Member: AAAI 2023, AAAI 2022, AAAI 2021.
  • Committee: NeurIPS 2022 virtual deep-dive session chair, CMU RISS Committee.

Design and source code from Jon's and Zhijian's website