Collaboration Challenges in Building ML-Enabled Systems: Communication, Documentation, Engineering, and Process

kaestner@cs.cmu.edu (N. Nahar, S. Zhou, G. Lewis, C. Kästner)
Tue, 31 May 2022 12:28:28 GMT

Nadia Nahar, Shurui Zhou, Grace Lewis, and Christian Kästner. Collaboration Challenges in Building ML-Enabled Systems: Communication, Documentation, Engineering, and Process. In Proceedings of the 44th International Conference on Software Engineering (ICSE), pages 413--425, New York, NY: ACM Press, May 2022. Distinguished Paper Award.

The introduction of machine learning (ML) components in software projects has created the need for software engineers to collaborate with data scientists and other specialists. While collaboration can always be challenging, ML introduces additional challenges with its exploratory model development process, additional skills and knowledge needed, difficulties testing ML systems, need for continuous evolution and monitoring, and non-traditional quality requirements such as fairness and explainability. Through interviews with 45 practitioners from 28 organizations, we identified key collaboration challenges that teams face when building and deploying ML systems into production. We report on common collaboration points in the development of production ML systems for requirements, data, and integration, as well as corresponding team patterns and challenges. We find that most of these challenges center around communication, documentation, engineering, and process and collect recommendations to address these challenges