Zhiqing Sun



PhD Student

CMU LTI (GHC 5507)


Hey there, welcome!

I am a Ph.D. student at CMU LTI, advised by Prof. Yiming Yang. I am generously supported by the Google PhD Fellowship in Natural Language Processing (2023), and be named one of UChicago’s 2023 Rising Stars in Data Science.

I received my M.S. degree in 2021 and anticipate completing my Ph.D. studies in late 2024. My academic journey began at Peking University, where I received a B.S. in Computer Science, advised by Prof. Zhi-Hong Deng.

Research Interests

I am generally interested in machine learning and natural language reasoning. My recent research focuses on aligning foundation models in a scalable manner. I am particularly interested in enhancing the reliability of foundation models, including large language models (LLMs) and large multimodal models (LMMs), through minimal human supervision and scalable oversight. This can be achieved using human-defined principles, factual feedback from real-world interactions, or recursive reward modeling. A few of my recent projects include:

  1. SALMON: Self-Alignment with Principle-Following Reward Models: We show that RLAIF can fully replace RLHF to align language models from scratch (enhancing both their alignment and capabilities), and build the world’s best open-source, non-distilled LLM for commercial use!
  2. Aligning Large Multimodal Models with Factually Augmented RLHF: We build the first open-source RLHF-trained LMM, and propose Factually Augmented RLHF (Fact-RLHF) that augments the reward model with additional factual information to alleviate the reward hacking phenomenon in RLHF.

I have also conducted research in related fields, including neural-symbolic reasoning (knowledge graphs), efficient sequence generation (non-autoregressive generation and sparse attention), model compression (knowledge distillation), and AI4Science (partial differential equations and combinatorial optimization).


  Language Technologies Institute, Carnegie Mellon University
  • Aug. 2019 - Present, M.S. / Ph.D. in Language Technologies
  School of Electrical Engineering & Computer Science (EECS), Peking University
  • Sept. 2015 - July 2019, B.S. in Computer Science (Summa Cum Laude)


  MIT-IBM Watson AI Lab
  Google Brain
  Google Brain
  Microsoft Research Asia
  Mila - Quebec Artificial Intelligence Institute & University of Montreal

Selected Publications

For a more complete list, see the publications page, or my google scholar page.

(*=equal contribution)


  1. NeurIPS (Spotlight)
    Principle-Driven Self-Alignment of Language Models from Scratch with Minimal Human Supervision
    Zhiqing Sun, Yikang Shen, Qinhong Zhou, Hongxin Zhang, Zhenfang Chen, David Cox, Yiming Yang, and Chuang Gan
    In Thirty-seventh Conference on Neural Information Processing Systems, 2023
  2. NeurIPS (Spotlight)
    DIFUSCO: Graph-based Diffusion Solvers for Combinatorial Optimization
    Zhiqing Sun, and Yiming Yang
    In Thirty-seventh Conference on Neural Information Processing Systems, 2023
  3. ICML
    A Neural PDE Solver with Temporal Stencil Modeling
    Zhiqing Sun, Yiming Yang, and Shinjae Yoo
    In Proceedings of the 40th International Conference on Machine Learning, 2023
  4. ICLR
    Recitation-Augmented Language Models
    Zhiqing Sun, Xuezhi Wang, Yi Tay, Yiming Yang, and Denny Zhou
    In The Eleventh International Conference on Learning Representations, 2023


  1. NeurIPS
    Dimes: A differentiable meta solver for combinatorial optimization problems
    Ruizhong Qiu*, Zhiqing Sun*, and Yiming Yang
    Advances in Neural Information Processing Systems, 2022
  2. ICLR
    Sparse attention with learning to hash
    Zhiqing Sun, Yiming Yang, and Shinjae Yoo
    In International Conference on Learning Representations, 2022


  1. ICCV
    Rethinking transformer-based set prediction for object detection
    Zhiqing Sun*, Shengcao Cao*, Yiming Yang, and Kris M Kitani
    In Proceedings of the IEEE/CVF international conference on computer vision, 2021


  1. ACL
    MobileBERT: a Compact Task-Agnostic BERT for Resource-Limited Devices
    Zhiqing Sun, Hongkun Yu, Xiaodan Song, Renjie Liu, Yiming Yang, and Denny Zhou
    In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020
  2. ICML
    An EM approach to non-autoregressive conditional sequence generation
    Zhiqing Sun, and Yiming Yang
    In International Conference on Machine Learning, 2020


  1. NeurIPS
    Fast structured decoding for sequence models
    Zhiqing Sun*, Zhuohan Li*, Haoqing Wang, Di He, Zi Lin, and Zhihong Deng
    Advances in Neural Information Processing Systems, 2019
  2. ICLR
    RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space
    Zhiqing Sun, Zhi-Hong Deng, Jian-Yun Nie, and Jian Tang
    In International Conference on Learning Representations, 2019