Software Engineering Thesis Proposal

  • Remote Access Enabled - Zoom
  • Virtual Presentation
  • Ph.D. Candidate
  • Ph.D. Program in Software Engineering
  • Institute for Software Research, Carnegie Mellon University
Thesis Proposals

Automated Testing of Robotic and Cyberphysical Systems

Robotics and cyberphysical systems are increasingly being deployed to settings where they are in frequent interaction with the public. Therefore, failures in these systems can be catastrophic by putting human lives in danger and causing extreme financial loss. Large-scale assessment of the quality of these systems before deployment can prevent these costly damages.

Because of the complexity and other special features of these systems, testing, and more specifically automated testing, faces challenges. In this thesis proposal, I study the unique challenges of testing robotics and cyberphysical systems by conducting a number of qualitative, quantitative, and mixed method studies, and propose an end-to-end automated testing pipeline to provide tools and methods that can help roboticists in large-scale automated testing of their systems. My key insight is that we can use (low-fidelity) simulation to automatically test robotic and cyberphysical systems, and identify many potentially catastrophic failures in advance at low cost.

My thesis statement is: Robotics and cyberphysical systems have unique features such as interacting with the physical world and integrating hardware and software components, which creates challenges for automated, large-scale testing approaches. Software-in-the-loop (low-fidelity) simulation can facilitate automated testing for these systems. Machine learning approaches (e.g., clustering) can be used to create an automated testing pipeline, which includes automated oracles and automated test input generation.

To support this statement, I propose the following work. In the preliminary work, which is already completed, I conducted a qualitative study and interviewed robotics practitioners about their testing practices and challenges. I identified nine main challenges roboticists face while testing their systems. In a case study on ArduPilot autonomous vehicle software, I investigated the potential impact of using low-fidelity software-based simulation on exposing failures in robotics systems, and showed that low-fidelity simulation can be an effective approach in detecting bugs and errors with low cost in robotic systems.

I propose to further study features in robotics simulators that are the most important for automated testing, and the challenges of using these simulators by conducting a large-scale survey with robotics practitioners.

I propose an approach to automatically generate oracles for cyberphysical systems using clustering, which can observe and identify common patterns of system behavior. These patterns can be used to distinguish erroneous behavior of the system and act as an oracle for an automated testing pipeline.

Finally, I propose to investigate automated test generation for these systems by first, identifying a suitable quality metric to evaluate the quality of test suites, and second, automatically generating test suites that target under-tested behaviors.

Thesis Committee:
Claire Le Goues (Chair)
Michael Hilton
Eunsuk Kang
John-Paul Ore (North Carolina State University)

Additional Proposal Information

Zoom Participation Enabled. See announcement.

For More Information, Please Contact: