Zhiqing Sun
孙之清
![avatar.jpg](/~zhiqings/assets/img/avatar.jpg?e10fd0b969768bdf2767bbf17cb9c0b5)
PhD Student
CMU LTI (GHC 5507)
zhiqings[at]cs.cmu.edu
Hey there, welcome!
I am a final-year Ph.D. candidate at CMU LTI, advised by Prof. Yiming Yang. My research is generously supported by the Google PhD Fellowship in Natural Language Processing (2023) and the OpenAI Superalignment Fast Grants (2024). I received my B.S. in Computer Science from Peking University.
Research Interests
I am generally interested in machine learning and artificial intelligence. My recent research focuses on scalable alignment of foundation models. I am particularly interested in enhancing the reliability of foundation models, including large language models (LLMs) and large multimodal models (LMMs), through minimal human supervision and scalable oversight. This can be achieved using human-defined principles, factual feedback from real-world interactions, or easy-to-hard generalization. A few of my recent projects include:
- Easy-to-Hard Generalization: Scalable Alignment Beyond Human Supervision: Guided by the observation that evaluation is easier than generation, we enabled large language models to excel on hard math problems beyond human evaluation capabilities through the easy-to-hard generalization of evaluators (e.g., process reward models).
- SALMON: Self-Alignment with Instructable Reward Models: We developed an Instructable Reward Model that helps RLAIF fully replace RLHF to align language models from scratch (enhancing both their alignment and capabilities)!
- Aligning Large Multimodal Models with Factually Augmented RLHF: We proposed Factually Augmented RLHF (Fact-RLHF) that augments the reward model with additional factual information to alleviate the reward hacking phenomenon in RLHF.
News
- Apr. 2024: Received the OpenAI Superalignment Fast Grants ($100,000) to support our research on easy-to-hard generalization.
- Mar. 2024: Gave a lecture about KG Embedding for the 11-741 course.
- Feb. 2024: Gave an invited talk at UIUC on scalable alignment.
- Jan. 2024: TAing for 11-741 Machine Learning with Graphs.
- Nov. 2023: Gave an invited talk at Caltech on scalable alignment, hosted by Prof. Yisong Yue
- Oct. 2023: Selected as the 2023 Rising Stars in Data Science and gave a talk on scalable alginment at the Rising Stars workshop in UChicago.
- Sept. 2023: Received the Microsoft Accelerate Foundation Models Research (AFMR) Initiative ($20,000).
- Sept. 2023: Received the Google PhD Fellowship in Natural Language Processing.
Education
Language Technologies Institute, Carnegie Mellon University
- Aug. 2019 - Present, M.S. / Ph.D. in Language Technologies
School of Electrical Engineering & Computer Science (EECS), Peking University
- Sept. 2015 - July 2019, B.S. in Computer Science (Summa Cum Laude)
Selected Publications
For a more complete list or preprints, see the publications page, or my google scholar page.
(*=equal contribution)
2024
2023
(Internship) Experience
- Allen Institute for Artificial Intelligence (AI2), Spring 2024
- MIT-IBM Watson AI Lab, Summer 2023
- Google Brain, Summer 2022
- Google Brain, Summer 2019
- Microsoft Research Asia, Spring 2019
- Mila & University of Montreal , Summer 2018