Joshua Sunshine

Associate Professor

Office: TCS Hall 323

I am an Associate Professor of Computer Science at Carnegie Mellon University. My home department is the Software and Societal Systems Department (S3D) and I have a courtesy appointment in the Human-Computer Interacation Institute.

I have broad research interests at the intersection of programming languages and human-computer interaction. I am particularly interested in better understanding of the factors that influence the usability of reusable software components.

I am fortunate to work with amazing students. See my advising page for details.

I direct the Research Experience for Undergraduates in interdisciplinary Software Engineering (REUSE) program with Charlie Garrod and Claire Le Goues. The program provides an opportunity for a diverse cohort of undergraduates from school across the country to spend the summer working with the world’s leading computer scientists. Students conduct research in a broad range of areas, including: collaboration and teams, computational statistics, computer architecture, diagrams and visualization, data mining and analysis, embedded systems, human-computer interaction, networking, privacy, program synthesis, program repair, programming languages, security, software architecture, societal computing, software requirements, and testing.

I completed my PhD in Software Engineering from Carnegie Mellon in December 2013. My dissertation focused on the usability of software libraries with ordering constraints (API protocols). I graduated from Brandeis University in 2004 and worked for almost four years as a software engineer before starting graduate school. I’m married to Kira Sunshine, and we have three children: Akiva, Elisheva, and Tamir.

selected publications

FSE

TerzoN: Human-in-the-Loop Software Testing with a Composite Oracle

Matthew C. Davis, Amy Wei, Brad A. Myers, and Joshua Sunshine

Proceedings of the ACM on Software Engineering, Volume 2, Issue FSE, 2025.

Abstract PDF

Software testing is difficult, tedious, and may consume 28%–50% of software engineering labor. Automatic test generators aim to ease this burden but have important trade-offs. Fuzzers use an implicit oracle that can detect obviously invalid results, but the oracle problem has no general solution, and an implicit oracle cannot automatically evaluate correctness. Test suite generators like EvoSuite use the program under test as the oracle and therefore cannot evaluate correctness. Property-based testing tools evaluate correctness, but users have difficulty coming up with properties to test and understanding whether their properties are correct. Consequently, practitioners create many test suites manually and often use an example-based oracle to tediously specify correct input and output examples. To help bridge the gaps among various oracle and tool types, we present the Composite Oracle, which organizes various oracle types into a hierarchy and renders a single test result per example execution. To understand the Composite Oracle’s practical properties, we built TerzoN, a test suite generator that includes a particular instantiation of the Composite Oracle. TerzoN displays all the test results in an integrated view composed from the results of three types of oracles and finds some types of test assertion inconsistencies that might otherwise lead to misleading test results. We evaluated TerzoN in a randomized controlled trial with 14 professional software engineers with a popular industry tool, fast-check, as the control. Participants using TerzoN elicited 72% more bugs (p < 0.01), accurately described more than twice the number of bugs (p < 0.01) and tested 16% more quickly (p < 0.05) relative to fast-check.
ICSE

A Study of Undefined Behavior Across Foreign Function Boundaries in Rust Libraries

Ian McCormack, Joshua Sunshine, and Jonathan Aldrich

In Proc. International Conference on Software Engineering (ICSE), 2025.

PDF
L@S

Edgeworth: Efficient and Scalable Authoring of Visual Thinking Activities

Wode Ni, Sam Estep, Hwei-Shin Harriman, Kenneth R. Koedinger, and Joshua Sunshine

In Proc. of the ACM Conference on Learning @ Scale, 2024. 🏆 Best Paper Nominee

Abstract PDF

Visual thinking with diagrams is a crucial skill for learning and problem-solving in STEM subjects. To improve in this area, students need a variety of visual problems for deliberate practice. However, in our interviews, educators shared that they struggle to create these practice exercises because of limitations of existing tools. We introduce Edgeworth, a tool designed to help educators easily create visual problems. Edgeworth works in two main ways: firstly, it takes a single diagram from the user and systematically alters it to produce many variations, which the educator can then choose from to create multiple problems. Secondly, it automates the layout of diagrams, ensuring consistent high quality without the need for manual adjustments. To assess Edgeworth, we carried out case studies, a technical evaluation, and expert walkthrough demonstrations. We show that Edgeworth can create problems in three domains: geometry, chemistry, and discrete math. These problems were authored in just 15 lines of Edgeworth code on average. Edgeworth generated usable answer options within the first 10 diagram variations in 87% of authored problems. Finally, educators gave positive feedback on Edgeworth’s utility and the real-world applicability of its outputs.
FSE

Nanofuzz: A usable tool for automatic test generation

Matthew Davis, Sangheon Choi, Sam Estep, Brad Myers, and Joshua Sunshine

In Proc. of the ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE), 2023.

Abstract PDF

In the United States alone, software testing labor is estimated to cost $48 billion USD per year. Despite widespread test execution automation and automation in other areas of software engineering, test suites continue to be created manually by software engineers. We have built a test generation tool, called NaNofuzz, that helps users find bugs in their code by suggesting tests where the output is likely indicative of a bug, e.g., that return NaN (not-a-number) values. NaNofuzz is an interactive tool embedded in a development environment to fit into the programmer’s workflow. NaNofuzz tests a function with as little as one button press, analyses the program to determine inputs it should evaluate, executes the program on those inputs, and categorizes outputs to prioritize likely bugs. We conducted a randomized controlled trial with 28 professional software engineers using NaNofuzz as the intervention treatment and the popular manual testing tool, Jest, as the control treatment. Participants using NaNofuzz on average identified bugs more accurately (p < .05, by 30%), were more confident in their tests (p < .03, by 20%), and finished their tasks more quickly (p < .007, by 30%).