Design practical AIs that can help users in complex tasks, where users are not oracle, and not static.

Recent Publications

2026
Comparing Developer and LLM Biases in Code Evaluation
Aditya Mittal, Ryan Shar, Zichu Wu, Shyam Agarwal, Tongshuang Wu, Chris Donahue, Ameet Talwalkar, Wayne Chi, Valerie Chen
COLM 2026
'I didn't Make the Micro Decisions': Measuring, Inducing, and Exposing Goal-Level AI Contributions in Collaboration
Eunsu Kim, Jessica R Mindel, Kyungjin Kim, Sherry Tongshuang Wu
COLM 2026
Mind the Sim2Real Gap in User Simulation for Agentic Tasks
Xuhui Zhou, Weiwei Sun, Qianou Ma, Yiqing Xie, Jiarui Liu, Weihua Du, Sean Welleck, Yiming Yang, Graham Neubig, Sherry Tongshuang Wu, Maarten Sap
COLM 2026
RECAP: An End-to-End Platform for Capturing, Replaying, and Analyzing AI-Assisted Programming Interactions
Keyu He, Qianou Ma, Wayne Chi, Valerie Chen, Tongshuang Wu
ACL Demo Track 2026
What Prompts Don’t Say: Understanding and Managing Underspecification in LLM Prompts
Chenyang Yang, Yike Shi, Qianou Ma, Michael Xieyang Liu, Christian Kästner, Tongshuang Wu
ACL Findings 2026
Scaling Collaborative Effort with Agents
Shannon Zejiang Shen, Valerie Chen, Ken Gu, Alexis Ross, Zixian Ma, Jillian Ross, Alex Gu, Chenglei Si, Wayne Chi, Andi Peng, Jocelyn J Shen, Ameet Talwalkar, David Sontag, Tongshuang Wu
ACL Findings 2026
Better Harnesses, Smaller Models: Building 90% Cheaper Agents via Automated Harness Adaptation
Chenyang Yang, Xinran Zhao, Tongshuang Wu, Christian Kästner
ArXiv 2026
Align AI to Dynamic Human-AI Workflows
Valerie Chen, Cleotilde Gonzalez, Anita Williams Woolley, Michael Lee, Tongshuang Wu, Vincent Conitzer, Aarti Singh
ArXiv 2026
DR Tulu: Reinforcement Learning with Evolving Rubrics for Deep Research
Rulin Shao, Akari Asai, Shannon Zejiang Shen, Hamish Ivison, Varsha Kishore, Jingming Zhuo, Xinran Zhao, Molly Park, Samuel G Finlayson, David Sontag, Tyler Murray, Sewon Min, Pradeep Dasigi, Luca Soldaini, Faeze Brahman, Wen-tau Yih, Tongshuang Wu, Luke Zettlemoyer, Yoon Kim, Hannaneh Hajishirzi, Pang Wei Koh
ICML 2026
Practice Less, Explain More: LLM-Supported Self-Explanation Improves Explanation Quality on Transfer Problems in Calculus
Eason Chen, Xinyi Tang, Yvonne Zhao, Meiyi Chen, Meryam Elmir, Elizabeth McLaughlin, Mingyu Yuan, Yumo Wang, Shyam Agarwal, Jared Cochrane, Jionghao Lin, Tongshuang Wu, Ken Koedinger
AIED 2026
“GenAI Defaults to Bias!” Gamify AI literacy through Reflections on Prompts
Qianou Ma, Megan Chai, Yike Tan, Jihun Choi, Jini Kim, Erik Harpstead, Geoff Kauffman, Tongshuang Wu
AIED 2026
The Empirically Grounded Adaptive Virtual Patient for Psychotherapy Training: Disclosure That Responds to Therapist Micro-Skills
Angela Chen, Siwei Jin, Catherine Bao, Canwen Wang, Robert E. Kraut, Tongshuang Wu, Haiyi Zhu
ArXiv 2026
Humans' ALMANAC: A Human Collaboration Dataset of Action-Level Mental Model Annotations for Agent Collaboration
Jiaju Chen, Yuxuan Lu, Jiayi Su, Chaoran Chen, Songlin Xiao, Zheng Zhang, Yun Wang, Yunyao Li, Jian Zhao, Tongshuang Wu, Toby Jia-Jun Li, Dakuo Wang, Bingsheng Yao
ArXiv 2026
SpeechEQ: Benchmarking Emotional Intelligence Quotient in Socially Aware Voice Conversational Models
Liang-Yuan Wu, Zih-Ching Chen, Tongshuang Wu, Chao-Han Huck Yang, Hua Shen
ArXiv 2026
Discretizing Reward Models
Vijay Viswanathan, Shiqi Wang, Devamanyu Hazarika, Chirag Nagpal, Tongshuang Wu, Graham Neubig, Yuning Mao
ArXiv 2026
OdysSim: Building Foundation Models for Human Behavior Simulation
Xuhui Zhou, Weiwei Sun, Weihua Du, Jiarui Liu, Haojia Sun, Qianou Ma, Tongshuang Wu, Yiming Yang, Maarten Sap
ArXiv 2026
Reinforcing Human Behavior Simulation via Verbal Feedback
Weiwei Sun, Xuhui Zhou, Jiarui Liu, Weihua Du, Haojia Sun, Yiqing Xie, Qianou Ma, Sihao Chen, Mengting Wan, Longqi Yang, Pei Zhou, Sherry Tongshuang Wu, Sean Welleck, Graham Neubig, Yiming Yang, Maarten Sap
ArXiv 2026
Behavioral Indicators of Overreliance During Interaction with Conversational Language Models
Chang Liu, Qinyi Zhou, Xinjie Shen, Xingyu Bruce Liu, Tongshuang Wu, Xiang 'Anthony' Chen
CHI 2026
Evidotes: Integrating Scientific Evidence and Anecdotes to Support Uncertainties Triggered by Peer Health Posts
Shreya Bali, Riku Arakawa, Peace Odiase, Tongshuang Wu, Mayank Goel
CHI 2026
Not Everyone Wins with LLMs: Behavioral Patterns and Pedagogical Implications in AI-assisted Data Analysis
Qianou Ma, Kenneth Koedinger, Tongshuang Wu
CHI 2026
Revela: Dense Retriever Learning via Language Modeling
Fengyu Cai, Tong Chen, Xinran Zhao, Sihao Chen, Hongming Zhang, Sherry Tongshuang Wu, Iryna Gurevych, Heinz Koeppl
ICLR 2026
Improving Attributed Long-form Question Answering with Intent Awareness
Xinran Zhao, Aakanksha Naik, Jay DeYoung, Joseph Chee Chang, Jena D. Hwang, Tongshuang Wu, Varsha Kishore
ICLR 2026
General Scales Unlock AI Evaluation with Explanatory and Predictive Power
Lexin Zhou, Lorenzo Pacchiardi, Fernando Martínez-Plumed, Katherine M. Collins, Yael Moros-Daval, Seraphina Zhang, Qinlin Zhao, Yitian Huang, Luning Sun, Jonathan E. Prunty, Zongqian Li, Pablo Sánchez-García, Kexin Jiang Chen, Pablo A. M. Casares, Jiyun Zu, John Burden, Behzad Mehrbakhsh, David Stillwell, Manuel Cebrian, Jindong Wang, Peter Henderson, Sherry Tongshuang Wu, Patrick C. Kyllonen, Lucy Cheke, Xing Xie, José Hernández-Orallo
Nature 2026
Empirical Modeling of Therapist-Client Dynamics in Psychotherapy Using LLM-Based Assessments
Angela Chen, Siwei Jin, Canwen Wang, Holly Swartz, Tongshuang Wu, Robert E Kraut, Haiyi Zhu
ArXiv 2026
Improving Automated Feedback Systems for Tutor Training in Low-Resource Scenarios through Data Augmentation
Chentianye Xu, Jionghao Lin, Tongshuang Wu, Vincent Aleven, Kenneth R. Koedinger
TLT 2026
Modeling Multi-Party Interaction in Couples Therapy: A Multi-Agent Simulation Approach
Canwen Wang, Angela Chen, Catherine Bao, Siwei Jin, Yee Kit Chan, Jessica R Mindel, Sijia Xie, Holly Swartz, Tongshuang Wu, Robert E Kraut, Haiyi Zhu
ArXiv 2026
2025
Checklists Are Better Than Reward Models For Aligning Language Models
Vijay Viswanathan, Yanchao Sun, Shuang Ma, Xiang Kong, Meng Cao, Graham Neubig, Tongshuang Wu
NeurIPS Spotlight 2025
MoR: Better Handling Diverse Queries with a Mixture of Sparse, Dense, and Human Retrievers
Jushaan Singh Kalra, Xinran Zhao, To Eun Kim, Fengyu Cai, Fernando Diaz, Tongshuang Wu
EMNLP 2025
cAST: Enhancing Code Retrieval-Augmented Generation with Structural Chunking via Abstract Syntax Tree
Yilin Zhang, Xinran Zhao, Zora Zhiruo Wang, Chenyang Yang, Jiayi Wei, Tongshuang Wu
EMNLP Findings 2025
The Ramon Llull's Thinking Machine for Automated Ideation
Xinran Zhao, Boyuan Zheng, Chenglei Si, Haofei Yu, Ken Ziyu Liu, Runlong Zhou, Ruochen Li, Tong Chen, Xiang Li, Yiming Zhang, Tongshuang Wu
COLM LM4Sci 2025
What Should We Engineer in Prompts? Training Humans in Requirement-Driven LLM Use
Qianou Ma, Weirui Peng, Chenyang Yang, Hua Shen, Kenneth Koedinger, Tongshuang Wu
TOCHI 2025
SPHERE: An Evaluation Card for Human-AI Systems
Qianou Ma*, Dora Zhao*, Xinran Zhao, Chenglei Si, Chenyang Yang, Ryan Louie, Ehud Reiter, Diyi Yang+, Tongshuang Wu+
ACL Findings 2025
RARE: Retrieval-Aware Robustness Evaluation for Retrieval-Augmented Generation Systems
Yixiao Zeng, Tianyu Cao, Danqing Wang, Xinran Zhao, Zimeng Qiu, Morteza Ziyadi, Tongshuang Wu, Lei Li
ArXiv 2025
LLMs as Workers in Human-Computational Algorithms? Replicating Crowdsourcing Pipelines with LLMs
Tongshuang Wu, Haiyi Zhu, Maya Albayrak, Alexis Axon, Amanda Bertsch, Wenxing Deng, Ziqi Ding, Bill Guo, Sireesh Gururaja, Tzu-Sheng Kuo, Jenny T Liang, Ryan Liu, Ihita Mandal, Jeremiah Milbauer, Xiaolin Ni, Namrata Padmanabhan, Subhashini Ramkumar, Alexis Sudjianto, Jordan Taylor, Ying-Jui Tseng, Patricia Vaidos, Zhijin Wu, Wei Wu, Chenyang Yang
CHI Case Study 2025
SOTOPIA-S4: a user-friendly system for flexible, customizable, and large-scale social simulation
Xuhui Zhou, Zhe Su, Sophie Feng, Jiaxu Zhou, Jen-tse Huang, Hsien-Te Kao, Spencer Lynch, Svitlana Volkova, Tongshuang Wu, Anita Woolley, Hao Zhu, Maarten Sap
NAACL Demo Track 2025
Orbit: A Framework for Designing and Evaluating Multi-objective Rankers
Chenyang Yang, Tesi Xiao, Michael Shavlovsky, Christian Kästner, Tongshuang Wu
IUI 2025
Evaluating Mathematical Reasoning Beyond Accuracy
Shijie Xia, Xuefeng Li, Yixin Liu, Tongshuang Wu, Pengfei Liu
AAAI 2025
2024
Tool Learning with Foundation Models
Yujia Qin, Shengding Hu, Yankai Lin, Weize Chen, Ning Ding, Ganqu Cui, Zheni Zeng, Yufei Huang, Chaojun Xiao, Chi Han, Yi Ren Fung, Yusheng Su, Huadong Wang, Cheng Qian, Runchu Tian, Kunlun Zhu, Shihao Liang, Xingyu Shen, Bokai Xu, Zhen Zhang, Yining Ye, Bowen Li, Ziwei Tang5, Jing Yi, Yuzhang Zhu, Zhenning Dai, Lan Yan, Xin Cong, Yaxi Lu, Weilin Zhao, Yuxiang Huang, Junxi Yan, Xu Han, Xian Sun, Dahai Li, Jason Phang, Cheng Yang, Tongshuang Wu, Heng Ji, Zhiyuan Liu, Maosong Sun
Computing Surveys 2024
HiMemFormer: Hierarchical Memory-Aware Transformer for Multi-Agent Action Anticipation
Zirui Wang, Xinran Zhao, Simon Stepputtis, Woojun Kim, Tongshuang Wu, Katia Sycara, Yaqi Xie
Video-Language Models Workshop @ NeurIPS 2024
Synthetic Multimodal Question Generation
Ian Wu, Sravan Jayanthi, Vijay Viswanathan, Simon Rosenberg, Sina Pakazad, Tongshuang Wu, Graham Neubig
EMNLP Findings 2024
What Is Wrong with My Model? Identifying Systematic Problems with Semantic Data Slicing
Chenyang Yang, Yining Hong, Grace A. Lewis, Tongshuang Wu, Christian Kästner
ASE 2024
Beyond Relevance: Evaluate and Improve Retrievers on Perspective Awareness
Xinran Zhao, Tong Chen, Sihao Chen, Hongming Zhang, Tongshuang Wu
CoLM 2024
Self-Guide: Better Task-Specific Instruction Following via Self-Synthetic Finetuning
Chenyang Zhao, Xueying Jia, Vijay Viswanathan, Graham Neubig, Tongshuang Wu
CoLM 2024
"Merge Conflicts!" Exploring the Impacts of External Distractors to Parametric Knowledge Graphs
Cheng Qian, Xinran Zhao, Tongshuang Wu
CoLM 2024
Do LLMs Exhibit Human-Like Response Biases? A Case Study in Survey Design
Lindia Tjuatja, Valerie Chen, Tongshuang Wu, Ameet Talwalkar, Graham Neubig
TACL 2024
Fact-and-Reflection (FaR) Improves Confidence Calibration of Large Language Models
Xinran Zhao, Hongming Zhang, Xiaoman Pan, Wenlin Yao, Dong Yu, Tongshuang Wu, Jianshu Chen
ACL Findings 2024
Better Synthetic Data by Retrieving and Transforming Existing Datasets
Saumya Gandhi, Ritu Gala, Vijay Viswanathan, Tongshuang Wu, Graham Neubig
ACL Findings 2024
A Large Scale Audit of Dataset Licensing and Attribution in AI
Shayne Longpre, Robert Mahari, Anthony Chen, Naana Obeng-Marnu, Damien Sileo, William Brannon, Niklas Muennighoff, Nathan Khazam, Jad Kabbara, Kartik Perisetla, Xinyi (Alexis) Wu, Enrico Shippole, Kurt Bollacker, Tongshuang Wu, Luis Villa, Sandy Pentland, Deb Roy, Sara Hooker
Nature Machine Intelligence 2024
How to Teach Programming in the AI Era? Using LLMs as a Teachable Agent for Debugging Best Paper, Best Interactive Event
Qiaomu Ma, Hua Shen, Kenneth Koedinger, Tongshuang Wu
AIED 2024
Generating Situated Reflection Triggers About Alternative Solution Paths: A Case Study in Generative AI for Computer-Supported Collaborative Learning Best Paper Nominee
Atharva Naik, Jessica Ruhan Yin, Anusha Kamath, Qianou Ma, Sherry Tongshuang Wu, Charles Murray, Majd Sakr, Carolyn P. Rose
AIED 2024
Large Language Models Help Humans Verify Truthfulness – Except When They are Convincingly Wrong
Chenglei Si, Navita Goyal, Tongshuang Wu, Chen Zhao, Shi Feng, Hal Daumé III, Jordan Boyd-Graber
NAACL 2024
Selenite: Scaffolding Online Sensemaking with Comprehensive Overviews Elicited from Large Language Models
Michael Xieyang Liu, Tongshuang Wu, Tianying Chen, Franklin Mingzhe Li, Aniket Kittur, Brad A. Myers
CHI 2024
Wikibench: Community-Driven Data Curation for AI Evaluation on Wikipedia
Tzu-Sheng Kuo, Aaron Halfaker, Zirui Cheng, Jiwoo Kim, Meng-Hsin Wu, Tongshuang Wu, Ken Holstein, Haiyi Zhu
CHI 2024
Large Language Models Enable Few-Shot Clustering
Vijay Viswanathan, Kiril Gashteovski, Carolin Lawrence, Tongshuang Wu, Graham Neubig
TACL 2024
2023
BiasX: "Thinking Slow" in Toxic Content Moderation with Explanations of Implied Social Biases
Yiming Zhang, Sravani Nanduri, Liwei Jiang, Tongshuang Wu, Maarten Sap
EMNLP 2023
From Nuisance to News Sense: Augmenting the News with Cross-document Evidence and Context
Jeremiah Milbauer, Ziqi Ding, Zhijin Wu, Tongshuang Wu
EMNLP Demo Track 2023
Promp2Model: Generating Deployable Models from Natural Language Instructions
Vijay Viswanathan, Chenyang Zhao, Amanda Bertsch, Tongshuang Wu, Graham Neubig
EMNLP Demo Track 2023
Beyond Testers' Biases: Guiding Model Testing with Knowledge Bases using LLMs
Chenyang Yang, Rishabh Rustogi, Rachel Brower-Sinning, Grace Lewis, Christian Kaestner, Tongshuang Wu
EMNLP Findings 2023
Bridging the Gap: A Survey on Integrating (Human) Feedback for Natural Language Generation
Patrick Fernandes, Aman Madaan, Emmy Liu, António Farinhas, Pedro Henrique Martins, Amanda Bertsch, José G. C. de Souza, Shuyan Zhou, Tongshuang Wu, Graham Neubig, André F. T. Martins
TACL 2023
Measuring Adversarial Datasets
Yuanchen Bai, Raoyi Huang, Vijay Viswanathan, Tzu-Sheng Kuo, Tongshuang Wu
AACL ART of Safety 2023
Synergi: A Mixed-Initiative System for Scholarly Synthesis and Sensemaking
Hyeonsu Kang, Tongshuang Wu, Joseph Chee Chang, Aniket Kittur
UIST 2023
DataFinder: Scientific Dataset Recommendation from Natural Language Descriptions
Vijay Viswanathan, Luyu Gao, Tongshuang Wu, Pengfei Liu, Graham Neubig
ACL 2023
Is AI the Better Programming Partner? Human-Human Pair Programming vs. Human-AI pAIr Programming
Qianou Christina Ma, Tongshuang Wu, Kenneth Koedinger
AIED2023 Empowering Education with LLMs 2023
Seeing Seeds Beyond Weeds: Green Teaming Generative AI for Beneficial Uses
Logan Stapleton, Jordan Taylor, Sarah Fox, Tongshuang Wu, Haiyi Zhu
ArXiv 2023
Capabilities for Better ML Engineering
Chenyang Yang, Rachel Brower-Sinning, Grace A. Lewis, Christian Kästner, Tongshuang Wu
AAAI SafeAI 2023

Current Members

Simret Araya Gebreegziabher (Postdoc) ,
co-advisor: Bob Kraut, Haiyi Zhu
Human-AI Co-pilot
Chenyang Yang (PhD) ,
co-advisor: Christian Kästner
Human-Centered ML Engineering
Vijay Viswanathan (PhD) ,
co-advisor: Graham Neubig
Synthesize Supervision Signals
Christina Ma (PhD) ,
co-advisor: Ken Koedinger
Train People for Future of Work
Xinran Zhao (PhD)
Information Seeking for Complex Tasks
Jessie Mindel (PhD)
Simulated Agents and Collective Sensemaking
Zheyuan Zhang (PhD)
Human-Agent Interaction
Eunsu Kim (PhD)
AI Contributions to Human Tasks.
Yuan Tian (Visit)
AI for Data Analysis

Alumni

Cheng Qian (Visit)
LLM hullucination. Now PhD student at UIUC.
Cassandra Shi (Undergrad)
Requirement-driven LLMs. Now PhD at NYU-Shanghai.
Shaan Lehal (Undergrad)
LLM sensemaking copilot
Yashika Batra (Undergrad)
LLM sensemaking copilot
Alina Chen (Undergrad)
LLM sensemaking copilot
Samriddhi Bhardwaj (Undergrad)
LLM sensemaking copilot
Alex Cheung (Undergrad)
LLM sensemaking copilot
Yuanchen (Sophie) Bai (Master)
NLP dataset characterization
Jushaan Kalra (Master)
Multi-domain Retrieval. Now software engineer at Snowflake.
Yilin Zhang (Master)
Code Retrieval with AST. Now software engineer at Google.
Callum Zhao (Master)
Build Effective Agentic Skills
Atharva Naik (Master)
LLM in CS education. Now PhD student at CMU.
Keyu He (Master)
Eval of Human Prompt Ability
Raoyi (Cathy) Huang (Master)
NLP dataset characterization. Now PhD student at Cornell.
Yiyang (Diana) Wang (Master)
End-User Prompt Disambiguation. Now PhD student at Georgia Tech.