Zhiqing Sun



PhD Student

CMU LTI (GHC 5507)


Hey there, welcome!

I am a final-year Ph.D. candidate at CMU LTI, advised by Prof. Yiming Yang. My research is generously supported by the Google PhD Fellowship in Natural Language Processing (2023) and the OpenAI Superalignment Fast Grants (2024). I received my B.S. in Computer Science from Peking University.

Research Interests

I am generally interested in machine learning and artificial intelligence. My recent research focuses on scalable alignment of foundation models. I am particularly interested in enhancing the reliability of foundation models, including large language models (LLMs) and large multimodal models (LMMs), through minimal human supervision and scalable oversight. This can be achieved using human-defined principles, factual feedback from real-world interactions, or easy-to-hard generalization. A few of my recent projects include:

  1. Easy-to-Hard Generalization: Scalable Alignment Beyond Human Supervision: Guided by the observation that evaluation is easier than generation, we enabled large language models to excel on hard math problems beyond human evaluation capabilities through the easy-to-hard generalization of evaluators (e.g., process reward models).
  2. SALMON: Self-Alignment with Instructable Reward Models: We developed an Instructable Reward Model that helps RLAIF fully replace RLHF to align language models from scratch (enhancing both their alignment and capabilities)!
  3. Aligning Large Multimodal Models with Factually Augmented RLHF: We proposed Factually Augmented RLHF (Fact-RLHF) that augments the reward model with additional factual information to alleviate the reward hacking phenomenon in RLHF.



  Language Technologies Institute, Carnegie Mellon University
  • Aug. 2019 - Present, M.S. / Ph.D. in Language Technologies
  School of Electrical Engineering & Computer Science (EECS), Peking University
  • Sept. 2015 - July 2019, B.S. in Computer Science (Summa Cum Laude)

Selected Publications

For a more complete list or preprints, see the publications page, or my google scholar page.

(*=equal contribution)


  1. ACL Findings
    Aligning Large Multimodal Models with Factually Augmented RLHF
    Zhiqing Sun*, Sheng Shen*, Shengcao Cao*, Haotian Liu, Chunyuan Li, Yikang Shen, Chuang Gan+, Liang-Yan Gui+, Yu-Xiong Wang+, Yiming Yang+, Kurt Keutzer+, and Trevor Darrell+
    In Findings of the Association for Computational Linguistics: ACL 2024, 2024
  2. ICLR
    SALMON: Self-Alignment with Instructable Reward Models
    Zhiqing Sun, Yikang Shen, Hongxin Zhang, Qinhong Zhou, Zhenfang Chen, David Daniel Cox, Yiming Yang, and Chuang Gan
    In The Twelfth International Conference on Learning Representations, 2024


  1. NeurIPS (Spotlight)
    Principle-Driven Self-Alignment of Language Models from Scratch with Minimal Human Supervision
    Zhiqing Sun, Yikang Shen, Qinhong Zhou, Hongxin Zhang, Zhenfang Chen, David Cox, Yiming Yang, and Chuang Gan
    In Thirty-seventh Conference on Neural Information Processing Systems, 2023
  2. NeurIPS (Spotlight)
    DIFUSCO: Graph-based Diffusion Solvers for Combinatorial Optimization
    Zhiqing Sun, and Yiming Yang
    In Thirty-seventh Conference on Neural Information Processing Systems, 2023
  3. ICLR
    Recitation-Augmented Language Models
    Zhiqing Sun, Xuezhi Wang, Yi Tay, Yiming Yang, and Denny Zhou
    In The Eleventh International Conference on Learning Representations, 2023

(Internship) Experience

  • Allen Institute for Artificial Intelligence (AI2), Spring 2024
  • MIT-IBM Watson AI Lab, Summer 2023
  • Google Brain, Summer 2022
  • Google Brain, Summer 2019
  • Microsoft Research Asia, Spring 2019
  • Mila & University of Montreal , Summer 2018