Experience from Measuring Program Comprehension -- Toward a General Framework

kaestner@cs.cmu.edu (J. Siegmund, C. Kästner, S. Apel, A. Brechmann, G. Saake)
Sat, 05 Mar 2016 18:41:39 -0500

Janet Siegmund, Christian Kästner, Sven Apel, André Brechmann, and Gunter Saake. Experience from Measuring Program Comprehension -- Toward a General Framework. In Proceedings of the Software Engineering 2013 -- Fachtagung des GI-Fachbereichs Softwaretechnik (SE), volume P-213 of Lecture Notes in Informatics, pages 239--257, Bonn, Germany: Gesellschaft für Informatik (GI), February 2013.

Program comprehension plays a crucial role during the software-development life cycle: Maintenance programmers spend most of their time with comprehending source code, and maintenance is the main cost factor in software development. Thus, if we can improve program comprehension, we can save considerable amount of time and cost. To improve program comprehension, we have to measure it first. However, program comprehension is a complex, internal cognitive process that we cannot observe directly. Typically, we need to conduct controlled experiments to soundly measure program comprehension. However, empirical research is applied only reluctantly in software engineering. To close this gap, we set out to support researchers in planning and conducting experiments regarding program comprehension. We report our experience with experiments that we conducted and present the resulting framework to support researchers in planning and conducting experiments. Additionally, we discuss the role of teaching for the empirical researchers of tomorrow.