Performance-Influence Models for Highly Configurable Systems

kaestner@cs.cmu.edu (N. Siegmund, A. Grebhahn, C. Kästner, S. Apel)
Sat, 05 Mar 2016 18:41:39 -0500

Norbert Siegmund, Alexander Grebhahn, Christian Kästner, and Sven Apel. Performance-Influence Models for Highly Configurable Systems. In Proceedings of the European Software Engineering Conference and ACM SIGSOFT Symposium on the Foundations of Software Engineering (ESEC/FSE), pages 284--294, New York, NY: ACM Press, August 2015.

Almost every complex software system today is configurable. While configurability has many benefits, it challenges performance prediction, optimization, and debugging. Often, the influences of the individual configurations options on performance is unknown. Worse, configuration options may interact, giving rise to a configuration space of possibly exponential size. Addressing this challenge, we propose an approach that derives a performance-influence model for a given configurable system, describing all relevant influences of configuration options and their interactions. Such a model shall be useful for automatic performance prediction and optimization, on the one hand, and performance debugging for developers, on the other hand. Our approach combines machine-learning and sampling technique in a novel way. Our approach improves over standard techniques in that it (1) represents influences of options and their interactions explicitly (which eases debugging), (2) smoothly integrates binary and numeric configuration options for the first time, (3) incorporates domain knowledge, if available (which eases learning and increases accuracy), (4) considers complex constraints among options, and (5) systematically reduces the solution space to a tractable size. A series of experiments demonstrates the feasibility of our approach in terms of the accuracy of the models learned as well as the accuracy of the performances predictions one can make with them. Using our approach, we were able to identify a number of real performance bugs and other problems in real-world systems.