Newsgroups: comp.ai.neural-nets,comp.software-eng,comp.software.testing
Path: cantaloupe.srv.cs.cmu.edu!fs7.ece.cmu.edu!europa.eng.gtefsd.com!library.ucla.edu!agate!boulder!news.coop.net!news.den.mmc.com!news2!pogo.den.mmc.com!hagar
From: hagar@pogo.den.mmc.com (Jon Duncan Hagar)
Subject: Using NN and Metrics to ID quality code 
Message-ID: <1994Oct6.134159.28184@news2.den.mmc.com>
Summary: NN can use measures to predict high risk code for testing 
Keywords: NN, metrics, high risk code, test 
Sender: news@news2.den.mmc.com (News Admin)
Nntp-Posting-Host: pogo.den.mmc.com
Organization: Martin Marietta Astronautics
Date: Thu, 6 Oct 1994 13:41:59 GMT
Lines: 44
Xref: glinda.oz.cs.cmu.edu comp.ai.neural-nets:19294 comp.software-eng:26371 comp.software.testing:2392

Several people have requested this info and several posting have 
dealt with metric questions and software quality, so I thought I'd 
do a posting.  In the International symposium on software 
reliability engineering (ISSRE) 93, Khoshgoftaar, Lanning, and 
Pandya reported results on using a Neural Network to identify 
high-risk (lower quality) code modules.  This type of classification 
can then be used to refine things like test approaches (I can know 
where to concentrate my test cases with more "pay-off").  The idea 
had been written about by others, notably Karunanithi , Whitley, 
and Malaiya in IEEE Software in July of 92.

The approach is to take a number of measures on software and the 
actual error reports levels to train a NN.  Then using measures on 
new software that does not have errors reported yet, to predict the 
likelihood that a piece of software may have an error (be at risk).  
Measures by themselves are criticized for being to simplistic and 
not revealing anything.  This technique attempts to combine a large 
number to gain more meaning.  Measures used include 
Operator/Operand count, unique operator/Operand count, 
Halsteads,Program volume, McCable's (cyclomatic and extended 
cyclomatic) complexity,  lines of code and executable lines of code.  
In the ISSRE article, NN results were compared to other analysis 
techniques and NN were shown to be better predictors of high risk 
code.

I was able to duplicate this work by using a simple ANN 
simulation tool.  When I trained a net, it revealed some interesting 
results that I later found were noted by other researches (for 
example code commenting density hasa relationship to code 
quality).  I do not see this approach as an end, but it can be used to 
reduce the volumes of measures some tools pour out and to indicate 
useful places in the code to concentrate testing on.  It is fairly 
simple to set up and train a NN (assuming you have data) as well 
as beinga good use of NN.

For those that are interested in more detail, I have contacts with 
some of the main authors of both the above sited papers, as well as 
the work I'm now doing.

-- 
Jon Hagar,  Mail Stop-H0512                  hagar@den.mmc.com
Martin Marietta Astronautics                (303) 977-1625
P.O. Box 179,   Denver, CO  80201       SD: Opinions are only mine
In the electronic village no matter where you go there you virtually are
