Differential Performance Fuzzing of Configuration Options

kaestner@cs.cmu.edu (H. Baik, C. Yang, V. Vikram, P. Jamshidi, R. Padhye, C. Kästner)
Mon, 13 Jul 2026 11:18:49 GMT

Haesue Baik, Chenyang Yang, Vasudev Vikram, Pooyan Jamshidi, Rohan Padhye, and Christian Kästner. Differential Performance Fuzzing of Configuration Options. In Proceedings of the International Workshop on Search-Based and Fuzz Testing (SBFT), pages 31--34, Los Alamitos, CA: IEEE Computer Society, April 2025.

Highly-configurable software often includes performance-sensitive configuration options. There are performance expectations across different configurations, but these expectations may not hold, due to inaccurate mental models, corner cases, or unanticipated interactions with other options. We propose differential performance fuzzing of configuration options, a fuzzing technique that uses differential performance feedback to automatically identify inputs that violate these expectations for specific configuration changes. By guiding fuzzing toward scenarios where a supposedly faster configuration performs worse, differential performance fuzzing reveals unexpected performance behavior effectively. In our preliminary evaluation, our method identified unexpected performance gains in configurations presumed slower for 4 configuration options in Closure, demonstrating the potential for detecting performance issues in real-world applications.