CMU LTI (GHC 5507)
zhiqings [at] cs.cmu.edu
Hey there, welcome!
I am a Ph.D. student in Language Technologies Institute, Carnegie Mellon University. I am fortunately advised by Prof. Yiming Yang. I received my M.S. degree in May 2021 and anticipate completing my Ph.D. studies in 2024. My academic journey began at Peking University, where I received a B.S. in Computer Science, advised by Prof. Zhi-Hong Deng.
My research focuses on neural-symbolic reasoning, which entails synergistically leveraging the strengths of machine learning systems with symbolic systems such as knowledge graphs and combinatorial optimization solvers.
More recently, my research interests have been oriented towards the alignment of Large Language Models (LLMs) and Large Multimodal Models (LMMs), with a special emphasis on improving reliability through scalable oversight (minimal human supervision) such as human-defined principles or factual feedback from real-world interactions.
I have also conducted research in related fields, including efficient sequence generation (non-autoregressive generation and sparse attention), model compression (knowledge distillation), and AI4Science (neural PDE solvers).
Language Technologies Institute, Carnegie Mellon University
- Aug. 2019 - Present, Ph.D. in Language Technologies
School of Electrical Engineering & Computer Science (EECS), Peking University
- Sept. 2015 - July 2019, B.S. in Computer Science (Summa Cum Laude)
MIT-IBM Watson AI Lab
- Apr. 2019 - Aug. 2019, Host: Denny Zhou
Microsoft Research Asia
- Oct. 2018 - Apr. 2019, Host: Dongdong Zhang
Mila - Quebec Artificial Intelligence Institute & University of Montreal
NeurIPS (Spotlight)Principle-Driven Self-Alignment of Language Models from Scratch with Minimal Human SupervisionIn Thirty-seventh Conference on Neural Information Processing Systems, 2023
NeurIPS (Spotlight)DIFUSCO: Graph-based Diffusion Solvers for Combinatorial OptimizationIn Thirty-seventh Conference on Neural Information Processing Systems, 2023
ICMLA Neural PDE Solver with Temporal Stencil ModelingIn Proceedings of the 40th International Conference on Machine Learning, 2023
ICLRRecitation-Augmented Language ModelsIn The Eleventh International Conference on Learning Representations, 2023
NeurIPSDimes: A differentiable meta solver for combinatorial optimization problemsAdvances in Neural Information Processing Systems, 2022
ICLRSparse attention with learning to hashIn International Conference on Learning Representations, 2022
ICCVRethinking transformer-based set prediction for object detectionIn Proceedings of the IEEE/CVF international conference on computer vision, 2021
ACLMobileBERT: a Compact Task-Agnostic BERT for Resource-Limited DevicesIn Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020
ICMLAn EM approach to non-autoregressive conditional sequence generationIn International Conference on Machine Learning, 2020
NeurIPSFast structured decoding for sequence modelsAdvances in Neural Information Processing Systems, 2019
ICLRRotatE: Knowledge Graph Embedding by Relational Rotation in Complex SpaceIn International Conference on Learning Representations, 2019