Eliminating Exception Handling Errors with Dependability Cases: A Comparative, Empirical Study
by Roy A. Maxion and Robert T. Olszewski
Abstract
Programs fail mainly for two reasons: logic errors in the code and
exception failures. Exception failures can account for up to two-thirds
of system crashes [1], hence, are worthy of serious attention.
Traditional approaches to reducing exception failures, such as code
reviews, walkthroughs, and formal testing, while very useful, are
limited in their ability to address a core problem: The programmer's
inadequate coverage of exceptional conditions. The problem of coverage
might be rooted in cognitive factors that impede the mental generation
(or recollection) of exception cases that would pertain in a particular
situation, resulting in insufficient software robustness. This paper
describes controlled experiments for testing the hypothesis that
robustness for exception failures can be improved through the use of
various coverage-enhancing techniques: N-version programming, group
collaboration, and dependability cases. N-version programming and
collaboration are well known. Dependability cases, derived from safety
cases, comprise a new methodology based on structured taxonomies and
memory aids for helping software designers think about and improve
exception handling coverage. All three methods showed improvements over
control conditions in increasing robustness to exception failures but
dependability cases proved most efficacious in terms of balancing cost
and effectiveness. A controlled experiment conducted with 119 subjects
revealed a statistically significant 34 percent increase (p < .01) in
exception handling robustness corresponding to use of dependability
cases. An ancillary experiment conducted with 53 subjects provided
convergent evidence that the effect is authentic and not due to
programming expertise alone.
Files
Last Modified: Wed Mar 14 15:40:34 EST 2001