Jielin Qiu
jielinq [at] cs (dot) cmu (dot) edu

I am a PhD student in the Computer Science Department at School of Computer Science, Carnegie Mellon University, advised by Prof. Ding Zhao. I also work closely with Prof. Bo Li and Prof. Douglas Weber. My research interests lie in Multimodal Machine Learning, I am interested in designing scalable inference and learning algorithms to connect language, perception, and actions for robust multimodal interaction. My current research lies in the foundations of multimodal learning with applications in multimedia, computer vision, natural language processing, cognition, and healthcare. My research is generously supported by DARPA, NSF, Allegheny Health Network, Cleveland Clinic, and Adobe.

Before coming to CMU, I received my B.Eng. from Shanghai Jiao Tong University, advised by Prof. Bao-Liang Lu.

Email  /  Google Scholar  /  LinkedIn

profile photo
Update
  • If you're an undergraduate or master student interested in RA/summer internship, please feel free to contact me. Our lab is looking for several students for research projects.
News
  • [2023-04] Invited talk at Microsoft Research Cambridge.
  • [2023-02] One paper accepted by CVPR 2023.
  • [2023-02] One paper accepted by ICASSP 2023.
  • [2023-01] Start a research internship at Microsoft.
  • [2023-01] One paper accepted by EACL 2023.
  • [2023-01] One paper accepted by AISTATS 2023.
  • [2022-10] One paper accepted by WACV 2023.
  • [2022-10] One paper accepted by NeurIPS 2022 Workshop on Distribution Shifts.
  • [2022-10] Top Reviewers in NeurIPS 2022.
  • [2022-06] One paper accepted by MLHC 2022.
  • [2022-05] Start a research internship at AWS AI.
  • [2022-05] One paper accepted by ICML 2022 workshop on Principles of Distribution Shift.
  • [2022-04] One paper accepted by ICLR 2022 Workshop on Socially Responsible Machine Learning.
  • [2021-09] Receive a gift funding from Adobe. Thanks, Adobe!
  • [2021-05] Start a research internship at Adobe research.
Preprints

* marked as equal contribution

Are Multimodal Models Robust to Image and Text Perturbations?
Jielin Qiu, Yi Zhu, Xingjian Shi, Florian Wenzel, Zhiqiang Tang, Ding Zhao, Bo Li, Mu Li
Under Review / arXiv / Project Webpage / code (coming soon)

Interpolation for Robust Learning: Data Augmentation on Geodesics
Jiacheng Zhu, Jielin Qiu, Aritra Guha, Zhuolin Yang, XuanLong Nguyen, Bo Li, Ding Zhao
Under Review / arXiv

Semantics-Consistent Cross-domain Summarization via Optimal Transport Alignment
Jielin Qiu, Jiacheng Zhu, Mengdi Xu, Franck Dernoncourt, Trung Bui, Zhaowen Wang, Bo Li, Ding Zhao, Hailin Jin
Under Review / arXiv

An Empirical Exploration of Cross-domain Alignment between Language and Electroencephalogram
William Han*, Jielin Qiu*, Jiacheng Zhu, Mengdi Xu, Douglas Weber, Bo Li, Ding Zhao
Under Review / arXiv / code

Selected Publications

* marked as equal contribution

Transfer Knowledge from Natural Language to Electrocardiography: Can We Detect Cardiovascular Disease Through Language Models?
Jielin Qiu*, William Han*, Jiacheng Zhu, Mengdi Xu, Michael Rosenberg, Emerson Liu, Douglas Weber, Ding Zhao
EACL 2023 / arXiv / code

Cardiac Disease Diagnosis on Imbalanced Electrocardiography Data Through Optimal Transport Augmentation
Jielin Qiu*, Jiacheng Zhu*, Mengdi Xu, Peide Huang, Michael Rosenberg, Douglas Weber, Emerson Liu, Ding Zhao
ICASSP 2023 / arXiv

Align and Attend: Multimodal Summarization with Dual Contrastive Losses
Bo He, Jun Wang, Jielin Qiu, Abhinav Shrivastava, Trung Bui, Zhaowen Wang
CVPR 2023 / arXiv / code

Benchmarking Robustness under Distribution Shift of Multimodal Image-Text Models
Jielin Qiu, Yi Zhu, Xingjian Shi, Zhiqiang Tang, Ding Zhao, Bo Li, Mu Li
NeurIPS 2022 Workshop on Distribution Shifts / Paper / Press / code (coming soon)

LiveSeg: Unsupervised Multimodal Temporal Segmentation of Long Livestream Videos
Jielin Qiu, Franck Dernoncourt, Trung Bui, Zhaowen Wang, Ding Zhao, Hailin Jin
WACV 2023 / arXiv / Press / code (coming soon)

Group Distributionally Robust Reinforcement Learning with Hierarchical Latent Variables
Mengdi Xu, Peide Huang, Yaru Niu, Visak Kumar, Jielin Qiu, Chao Fang, Kuan-Hui Lee, Xuewei Qi, Henry Lam, Bo Li, Ding Zhao
AISTATS 2023 / paper / code

GeoECG: Data Augmentation via Wasserstein Geodesic Perturbation for Robust Electrocardiogram Prediction
Jiacheng Zhu*, Jielin Qiu*, Zhuolin Yang, Douglas Weber, Michael Rosenberg, Emerson Liu, Bo Li, Ding Zhao
MLHC 2022 / arXiv

Data Augmentation via Wasserstein Geodesic Perturbation for Robust Electrocardiogram Prediction
Jiacheng Zhu*, Jielin Qiu*, Zhuolin Yang, Michael Rosenberg, Emerson Liu, Bo Li, Ding Zhao
ICLR 2022 Workshop on Socially Responsible Machine Learning (SRML) / Paper

Comparing Recognition Performance and Robustness of Multimodal Deep Learning Models for Multimodal Emotion Recognition
Wei Liu, Jielin Qiu, Wei-Long Zheng, Bao-Liang Lu
IEEE Transactions on Cognitive and Developmental Systems 2021 / Paper / arXiv / code

Visual Sequence Learning in Hierarchical Prediction Networks and Primate Visual Cortex
Jielin Qiu, Ge Huang, Tai Sing Lee
NeurIPS 2019 / Paper

Investigating Sex Differences in Classification of Five Emotions from EEG and Eye Movement Signals
Lan-Qing Bao, Jielin Qiu, Hao Tang, Wei-Long Zheng, Bao-Liang Lu
EMBC 2019 / Paper / code

Approximation Gradient Error Variance Reduced Optimization
Weiye Zhao, Yang Liu, Xiaoming Zhao, Jielin Qiu, Jian Peng
AAAI 2019 / Paper

Multi-view Emotion Recognition Using Deep Canonical Correlation Analysis
Jielin Qiu, Wei Liu, Bao-Liang Lu
ICONIP 2018 / Paper / code

Teaching
Services
  • Reviewer: CVPR 2023, ICML 2023, ICCV 2023, KDD 2023, EACL 2023, MICCAI 2023, ICASSP 2023, WACV 2023, AISTATS 2023, MLHC 2023, CHIL 2023, NeurIPS 2022, CVPR 2022, ICML 2022, ECCV 2022, ACM MM 2022, MLHC 2022, CHIL 2022, ICML 2021.
  • PC Member: ACL 2023, AAAI 2023, EMNLP 2022, AAAI 2022, AAAI 2021.
  • Committee: NeurIPS 2022 virtual deep-dive session chair, CMU RISS Committee.

Design and source code from Jon's and Zhijian's website