ConfigCrusher: Towards White-Box Performance Analysis for Configurable Systems

kaestner@cs.cmu.edu (M. Velez, P. Jamshidi, F. Sattler, N. Siegmund, S. Apel, C. Kästner)
Fri, 25 Dec 2020 15:26:39 GMT

Miguel Velez, Pooyan Jamshidi, Florian Sattler, Norbert Siegmund, Sven Apel, and Christian Kästner. ConfigCrusher: Towards White-Box Performance Analysis for Configurable Systems. Automated Software Engineering -- An International Journal (AUSE), 27:265--300, 2020.

In configurable software systems, stakeholders are often interested in knowing how configuration options influence the performance of a system to facilitate, for example, the debugging and optimization processes of these systems. There are several black-box approaches to obtain this information, but they either sample the system end-to-end with a large number of configurations to make accurate predictions or miss important performance-influencing interactions when sampling few configurations. In addition, these approaches cannot pinpoint the parts of a program that are responsible for performance differences among configurations. This paper proposes ConfigCrusher, a white-box performance analysis that analyzes the implementation of a system to guide the performance analysis and exploits several insights about configurable systems in the process. ConfigCrusher employs a static data-flow analysis to identify how configuration options may influence control-flow decisions and instruments code regions corresponding to these decisions to dynamically analyze the influence of configuration options on the regions' performance. Our evaluation shows the feasibility of our white-box approach to more efficiently build performance models that are similar to or more accurate than current state-of-the-art approaches on 10 configurable systems. Overall, we showcase the benefits of white-box performance analyses and their potential to outperform black-box approaches and provide additional information for analyzing configurable systems.