Design practical AIs that can help users in complex tasks, where users are not oracle, and not static.

Recent Publications

2025
What Prompts Don’t Say: Understanding and Managing Underspecification in LLM Prompts
Chenyang Yang, Yike Shi, Qianou Ma, Michael Xieyang Liu, Christian Kästner, Tongshuang Wu
ArXiv 2025
Improving Automated Feedback Systems for Tutor Training in Low-Resource Scenarios through Data Augmentation
Chentianye Xu, Jionghao Lin, Tongshuang Wu, Vincent Aleven, Kenneth R. Koedinger
ArXiv 2025
Revela: Dense Retriever Learning via Language Modeling
Fengyu Cai, Tong Chen, Xinran Zhao, Sihao Chen, Hongming Zhang, Sherry Tongshuang Wu, Iryna Gurevych, Heinz Koeppl
ArXiv 2025
RARE: Retrieval-Aware Robustness Evaluation for Retrieval-Augmented Generation Systems
Yixiao Zeng, Tianyu Cao, Danqing Wang, Xinran Zhao, Zimeng Qiu, Morteza Ziyadi, Tongshuang Wu, Lei Li
ArXiv 2025
From Prompts to Reflection: Designing Reflective Play for GenAI Literacy
Qianou Ma, Megan Chai, Yike Tan, Jihun Choi, Jini Kim, Erik Harpstead, Geoff Kauffman, Tongshuang Wu
ArXiv 2025
Not Everyone Wins with LLMs: Behavioral Patterns and Pedagogical Implications in AI-assisted Data Analysis
Qianou Ma, Kenneth Koedinger, Tongshuang Wu
ArXiv 2025
DR Tulu: Reinforcement Learning with Evolving Rubrics for Deep Research
Rulin Shao, Akari Asai, Shannon Zejiang Shen, Hamish Ivison, Varsha Kishore, Jingming Zhuo, Xinran Zhao, Molly Park, Samuel G Finlayson, David Sontag, Tyler Murray, Sewon Min, Pradeep Dasigi, Luca Soldaini, Faeze Brahman, Wen-tau Yih, Tongshuang Wu, Luke Zettlemoyer, Yoon Kim, Hannaneh Hajishirzi, Pang Wei Koh
ArXiv 2025
Completion ≠ Collaboration: Scaling Collaborative Effort with Agents Best Paper
Shannon Zejiang Shen, Valerie Chen, Ken Gu, Alexis Ross, Zixian Ma, Jillian Ross, Alex Gu, Chenglei Si, Wayne Chi, Andi Peng, Jocelyn J Shen, Ameet Talwalkar, David Sontag, Tongshuang Wu
NeurIPS Responsible Foundation Model Workshop 2025
General Scales Unlock AI Evaluation with Explanatory and Predictive Power
Lexin Zhou, Lorenzo Pacchiardi, Fernando Martínez-Plumed, Katherine M. Collins, Yael Moros-Daval, Seraphina Zhang, Qinlin Zhao, Yitian Huang, Luning Sun, Jonathan E. Prunty, Zongqian Li, Pablo Sánchez-García, Kexin Jiang Chen, Pablo A. M. Casares, Jiyun Zu, John Burden, Behzad Mehrbakhsh, David Stillwell, Manuel Cebrian, Jindong Wang, Peter Henderson, Sherry Tongshuang Wu, Patrick C. Kyllonen, Lucy Cheke, Xing Xie, José Hernández-Orallo
Nature 2025
What Should We Engineer in Prompts? Training Humans in Requirement-Driven LLM Use
Qianou Ma, Weirui Peng, Chenyang Yang, Hua Shen, Kenneth Koedinger, Tongshuang Wu
TOCHI 2025
MoR: Better Handling Diverse Queries with a Mixture of Sparse, Dense, and Human Retrievers
Jushaan Singh Kalra, Xinran Zhao, To Eun Kim, Fengyu Cai, Fernando Diaz, Tongshuang Wu
EMNLP 2025
LLMs as Workers in Human-Computational Algorithms? Replicating Crowdsourcing Pipelines with LLMs
Tongshuang Wu, Haiyi Zhu, Maya Albayrak, Alexis Axon, Amanda Bertsch, Wenxing Deng, Ziqi Ding, Bill Guo, Sireesh Gururaja, Tzu-Sheng Kuo, Jenny T Liang, Ryan Liu, Ihita Mandal, Jeremiah Milbauer, Xiaolin Ni, Namrata Padmanabhan, Subhashini Ramkumar, Alexis Sudjianto, Jordan Taylor, Ying-Jui Tseng, Patricia Vaidos, Zhijin Wu, Wei Wu, Chenyang Yang
CHI Case Study 2025
SPHERE: An Evaluation Card for Human-AI Systems
Qianou Ma*, Dora Zhao*, Xinran Zhao, Chenglei Si, Chenyang Yang, Ryan Louie, Ehud Reiter, Diyi Yang+, Tongshuang Wu+
ACL Findings 2025
SOTOPIA-S4: a user-friendly system for flexible, customizable, and large-scale social simulation
Xuhui Zhou, Zhe Su, Sophie Feng, Jiaxu Zhou, Jen-tse Huang, Hsien-Te Kao, Spencer Lynch, Svitlana Volkova, Tongshuang Wu, Anita Woolley, Hao Zhu, Maarten Sap
NAACL Demo Track 2025
Evaluating Mathematical Reasoning Beyond Accuracy
Shijie Xia, Xuefeng Li, Yixin Liu, Tongshuang Wu, Pengfei Liu
AAAI 2025
Orbit: A Framework for Designing and Evaluating Multi-objective Rankers
Chenyang Yang, Tesi Xiao, Michael Shavlovsky, Christian Kästner, Tongshuang Wu
IUI 2025
cAST: Enhancing Code Retrieval-Augmented Generation with Structural Chunking via Abstract Syntax Tree
Yilin Zhang, Xinran Zhao, Zora Zhiruo Wang, Chenyang Yang, Jiayi Wei, Tongshuang Wu
EMNLP Findings 2025
Checklists Are Better Than Reward Models For Aligning Language Models
Vijay Viswanathan, Yanchao Sun, Shuang Ma, Xiang Kong, Meng Cao, Graham Neubig, Tongshuang Wu
NeurIPS Spotlight 2025
The Ramon Llull's Thinking Machine for Automated Ideation
Xinran Zhao, Boyuan Zheng, Chenglei Si, Haofei Yu, Ken Ziyu Liu, Runlong Zhou, Ruochen Li, Tong Chen, Xiang Li, Yiming Zhang, Tongshuang Wu
COLM LM4Sci 2025
2024
Do LLMs Exhibit Human-Like Response Biases? A Case Study in Survey Design
Lindia Tjuatja, Valerie Chen, Tongshuang Wu, Ameet Talwalkar, Graham Neubig
TACL 2024
Tool Learning with Foundation Models
Yujia Qin, Shengding Hu, Yankai Lin, Weize Chen, Ning Ding, Ganqu Cui, Zheni Zeng, Yufei Huang, Chaojun Xiao, Chi Han, Yi Ren Fung, Yusheng Su, Huadong Wang, Cheng Qian, Runchu Tian, Kunlun Zhu, Shihao Liang, Xingyu Shen, Bokai Xu, Zhen Zhang, Yining Ye, Bowen Li, Ziwei Tang5, Jing Yi, Yuzhang Zhu, Zhenning Dai, Lan Yan, Xin Cong, Yaxi Lu, Weilin Zhao, Yuxiang Huang, Junxi Yan, Xu Han, Xian Sun, Dahai Li, Jason Phang, Cheng Yang, Tongshuang Wu, Heng Ji, Zhiyuan Liu, Maosong Sun
Computing Surveys 2024
A Large Scale Audit of Dataset Licensing and Attribution in AI
Shayne Longpre, Robert Mahari, Anthony Chen, Naana Obeng-Marnu, Damien Sileo, William Brannon, Niklas Muennighoff, Nathan Khazam, Jad Kabbara, Kartik Perisetla, Xinyi (Alexis) Wu, Enrico Shippole, Kurt Bollacker, Tongshuang Wu, Luis Villa, Sandy Pentland, Deb Roy, Sara Hooker
Nature Machine Intelligence 2024
Synthetic Multimodal Question Generation
Ian Wu, Sravan Jayanthi, Vijay Viswanathan, Simon Rosenberg, Sina Pakazad, Tongshuang Wu, Graham Neubig
EMNLP Findings 2024
Large Language Models Help Humans Verify Truthfulness – Except When They are Convincingly Wrong
Chenglei Si, Navita Goyal, Tongshuang Wu, Chen Zhao, Shi Feng, Hal Daumé III, Jordan Boyd-Graber
NAACL 2024
Fact-and-Reflection (FaR) Improves Confidence Calibration of Large Language Models
Xinran Zhao, Hongming Zhang, Xiaoman Pan, Wenlin Yao, Dong Yu, Tongshuang Wu, Jianshu Chen
ACL Findings 2024
Better Synthetic Data by Retrieving and Transforming Existing Datasets
Saumya Gandhi, Ritu Gala, Vijay Viswanathan, Tongshuang Wu, Graham Neubig
ACL Findings 2024
Generating Situated Reflection Triggers About Alternative Solution Paths: A Case Study in Generative AI for Computer-Supported Collaborative Learning
Atharva Naik, Jessica Ruhan Yin, Anusha Kamath, Qianou Ma, Sherry Tongshuang Wu, Charles Murray, Majd Sakr, Carolyn P. Rose
AIED 2024
How to Teach Programming in the AI Era? Using LLMs as a Teachable Agent for Debugging Best Paper
Qiaomu Ma, Hua Shen, Kenneth Koedinger, Tongshuang Wu
AIED 2024
Wikibench: Community-Driven Data Curation for AI Evaluation on Wikipedia
Tzu-Sheng Kuo, Aaron Halfaker, Zirui Cheng, Jiwoo Kim, Meng-Hsin Wu, Tongshuang Wu, Ken Holstein, Haiyi Zhu
CHI 2024
Self-Guide: Better Task-Specific Instruction Following via Self-Synthetic Finetuning
Chenyang Zhao, Xueying Jia, Vijay Viswanathan, Graham Neubig, Tongshuang Wu
CoLM 2024
"Merge Conflicts!" Exploring the Impacts of External Distractors to Parametric Knowledge Graphs
Cheng Qian, Xinran Zhao, Tongshuang Wu
CoLM 2024
Beyond Relevance: Evaluate and Improve Retrievers on Perspective Awareness
Xinran Zhao, Tong Chen, Sihao Chen, Hongming Zhang, Tongshuang Wu
CoLM 2024
Selenite: Scaffolding Online Sensemaking with Comprehensive Overviews Elicited from Large Language Models
Michael Xieyang Liu, Tongshuang Wu, Tianying Chen, Franklin Mingzhe Li, Aniket Kittur, Brad A. Myers
CHI 2024
What Is Wrong with My Model? Identifying Systematic Problems with Semantic Data Slicing
Chenyang Yang, Yining Hong, Grace A. Lewis, Tongshuang Wu, Christian Kästner
ASE 2024
HiMemFormer: Hierarchical Memory-Aware Transformer for Multi-Agent Action Anticipation
Zirui Wang, Xinran Zhao, Simon Stepputtis, Woojun Kim, Tongshuang Wu, Katia Sycara, Yaqi Xie
Video-Language Models Workshop @ NeurIPS 2024
2023
Bridging the Gap: A Survey on Integrating (Human) Feedback for Natural Language Generation
Patrick Fernandes, Aman Madaan, Emmy Liu, António Farinhas, Pedro Henrique Martins, Amanda Bertsch, José G. C. de Souza, Shuyan Zhou, Tongshuang Wu, Graham Neubig, André F. T. Martins
TACL 2023
Large Language Models Enable Few-Shot Clustering
Vijay Viswanathan, Kiril Gashteovski, Carolin Lawrence, Tongshuang Wu, Graham Neubig
TACL 2023
BiasX: "Thinking Slow" in Toxic Content Moderation with Explanations of Implied Social Biases
Yiming Zhang, Sravani Nanduri, Liwei Jiang, Tongshuang Wu, Maarten Sap
EMNLP 2023
Beyond Testers' Biases: Guiding Model Testing with Knowledge Bases using LLMs
Chenyang Yang, Rishabh Rustogi, Rachel Brower-Sinning, Grace Lewis, Christian Kaestner, Tongshuang Wu
EMNLP Findings 2023
Synergi: A Mixed-Initiative System for Scholarly Synthesis and Sensemaking
Hyeonsu Kang, Tongshuang Wu, Joseph Chee Chang, Aniket Kittur
UIST 2023
Promp2Model: Generating Deployable Models from Natural Language Instructions
Vijay Viswanathan, Chenyang Zhao, Amanda Bertsch, Tongshuang Wu, Graham Neubig
EMNLP Demo Track 2023
From Nuisance to News Sense: Augmenting the News with Cross-document Evidence and Context
Jeremiah Milbauer, Ziqi Ding, Zhijin Wu, Tongshuang Wu
EMNLP Demo Track 2023
Seeing Seeds Beyond Weeds: Green Teaming Generative AI for Beneficial Uses
Logan Stapleton, Jordan Taylor, Sarah Fox, Tongshuang Wu, Haiyi Zhu
ArXiv 2023
DataFinder: Scientific Dataset Recommendation from Natural Language Descriptions
Vijay Viswanathan, Luyu Gao, Tongshuang Wu, Pengfei Liu, Graham Neubig
ACL 2023
Capabilities for Better ML Engineering
Chenyang Yang, Rachel Brower-Sinning, Grace A. Lewis, Christian Kästner, Tongshuang Wu
AAAI SafeAI 2023
Measuring Adversarial Datasets
Yuanchen Bai, Raoyi Huang, Vijay Viswanathan, Tzu-Sheng Kuo, Tongshuang Wu
AACL ART of Safety 2023
Is AI the Better Programming Partner? Human-Human Pair Programming vs. Human-AI pAIr Programming
Qianou Christina Ma, Tongshuang Wu, Kenneth Koedinger
AIED2023 Empowering Education with LLMs 2023

Current Members

Chenyang Yang (PhD) ,
co-advisor: Christian Kästner
Human-Centered ML Engineering
Vijay Viswanathan (PhD) ,
co-advisor: Graham Neubig
Synthetic data generation
Christina Ma (PhD) ,
co-advisor: Ken Koedinger
Preparing Students for Effective Human-LLM Partnerships
Xinran Zhao (PhD)
Information Seeking and Retrieval for Complex Tasks
Jessie Mindel (PhD)
Simulated Agents and Collective Sensemaking
Zheyuan Zhang (PhD)
Human-agent interaction
Jushaan Kalra (Master)
Multi-domain Retrieval
Keyu He (Master)
Multi-domain Retrieval
Yilin Zhang (Master)
Code Retrieval with AST
Cassandra Shi (Undergrad)
Requirement-driven LLMs
Eunsu Kim (Visiting)
Eval on AI contribution to human tasks.

Alumni

Cheng Qian (Visit)
LLM hullucination. Now PhD student at UIUC.
Alex Cheung (Undergrad)
LLM sensemaking copilot
Samriddhi Bhardwaj (Undergrad)
LLM sensemaking copilot
Shaan Lehal (Undergrad)
LLM sensemaking copilot
Yashika Batra (Undergrad)
LLM sensemaking copilot
Alina Chen (Undergrad)
LLM sensemaking copilot
Yiyang (Diana) Wang (Master)
End-User Prompt Disambiguation. Now PhD student at Georgia Tech.
Atharva Naik (Master)
LLM in CS education. Now PhD student at CMU.
Raoyi (Cathy) Huang (Master)
NLP dataset characterization. Now PhD student at Cornell.
Yuanchen (Sophie) Bai (Master)
NLP dataset characterization