The Linguistics of Problem Reports
Are there linguistic trends in people's natural language descriptions of software problems? To find out, we analyzed nearly 200,000 open source bug reports, specifically studying their one-line summaries. We found that each summary identified some software entity or entity behavior, and described its inadequacy and the context of its inadequacy. We also found a great deal of regularity in the structure of these descriptions, driven by the use of conjunctions and prepositions to relate the entities and behaviors to their context and inadequacy. We use these trends to identify new types of analyses that might be performed on reports in order to streamline triage, as well as identify design guidelines for tools that solicit descriptions of software problems from people.
this paper is in progress.
What are the productivity bottlenecks in programmers' maintenance tasks? Do answer this question, we studied several expert Java programmers using Eclipse to fix bugs and add features to a painting application. The study suggests that maintenance work consists of three activities: (1) forming a working set of task-relevant code fragments; (2) navigating the dependencies within this working set; and (3) repairing or creating the necessary code. The study identified several trends in these activities, including that developer's spent an average of 35% of their time scrolling within files and switching between files, to code they had already seen but needed to reference again.
Ko, A. J., Aung, H., and Myers, B. A. (2005). Eliciting Design Requirements for Maintenance-Oriented IDEs: A Detailed Study of Corrective and Perfective Maintenance Tasks. 27th International Conference on Software Engineering
, St. Louis, MI, May 15-21, 126-135.
[ local ]
[ acm ]
If you're curious, you can view the source code for the Paint application used in the study.
What determines a programmer's interruptibility? James Fogarty is leading an important line of research to build statistical models of people's interruptibility. We collaborated with James to focus specifically on programmer's interruptibility, and ran a lab study of dozens of Java programmers, interrupting them at regular intervals similar to those found in software development workplaces. Using a measure of interruptibility based on the duration of time before programmers acknowledged the interruptions, we found that the best predictors of their interruptibility were whether or not they were in the process of editing code. We concluded that, in general, any time programmers are externalizing information from their working memory is a bad time to interrupt.
Fogarty, J., Ko, A.J., Aung, H.H., Golden, E., Tang, K.P. and Hudson, S.E. (2005). Examining Task Engagement in Sensor-Based Statistical Models of Human Interruptibility. CHI 2005: Human Factors in Computing Systems
, Portland OR, April 2-7, 2005, 331-340
[ local ]
[ acm ]
Text Editing Strategies
How do programmers edit textual code (as opposed to visual or otherwise)? We did a fine grained analysis of several programmers character-by-character edits, finding that programmers generally preserve the syntactic integrity of their code. Almost all edits, however, require them to pass through some syntactically invalid state momentarily. We also found that programmers selected code in an editor structurally, but often desired multiple structural selections.
Ko, A. J., Aung, H., and Myers, B. A. (2005). Design Requirements for More Flexible Structured Editors from a Study of Programmers' Text Editing. Extended Abstracts CHI 2005: Human Factors in Computing Systems
, Portland OR, April 2-7, 2005, 1557-1560.
[ local ]
[ acm ]
What are the major barriers to people learning to program in modern programming environments? We studied a classroom full of non-programmers attempting to learn Visual Basic.NET and found that, while many barriers were difficulty with syntax errors, many others had to do with searching for, using, coordinating, and debugging uses of abstractions in APIs. We identified many ways that programming environments might lower or remove these barriers.
Ko, A. J. Myers, B. A., and Aung, H. (2004). Six Learning Barriers in End-User Programming Systems. IEEE Symposium on Visual Languages and Human-Centric Computing
, Rome, Italy, September 26-29, 199-206.
[ local ]
[ ieee ]
Causes of Software Errors
What causes software errors? Traditional perspectives blame the programmers or the languages. We observed and documented groups developing interactive 3D simulations, the errors introduced into their code, and their efforts to find and repair these errors. Programmers began every debugging task with a question such as "Why did..." or "Why didn't...". These questions implicitly made several false assumptions about what their applications were doing at runtime, leading to dead ends in debugging efforts. About half of the errors introduced occurred as a result of efforts to repair other errors that were based on these false assumptions. Our journal article on this work also gives a thorough overview of research on human error, placed in the context of software development, and introduces a framework for describing the causes of software errors as coming from multiple sources.
Ko, A. J. and Myers, B. A. (2005). A Framework and Methodology for Studying the Causes of Software Errors in Programming Systems. Journal of Visual Languages and Computing
, 16, 1-2, 41-84.
[ local ]
[ elsevier ]