Software engineering does not as yet have well-established norms for validating research results. The range of problems and solutions is wide enough that no single technique will suffice -- but validation is certainly required. The appropriate validation technique(s) depends both on characteristics of the problem and characteristics of the solution. The alternatives are laid out below; first we need a little notation.
A typical research project begins with a problem in the world, PW, that serves to motivate the research. The researcher translates this to a problem, PM, in a model setting, M. In some cases the model setting is explicit or even shared within a line of research; in other cases it is implicit in the problem statement. The researcher produces a solution, SM to PM. This may include developing a tool or technique, TM and demonstrating the solution on examples EM, all in the model setting. Validation consists of showing first that SM solves PM and then that SM provides guidance for developing a solution SW that works for PW in the world at large. This last step -- the end-to-end check on the work -- is altogether too often omitted.
A characteristic of software engineering research is that PM is as faithful as possible to the world, retaining the thorny difficulties and warts that actually made the problem in the world hard.
<<<<<what remains to be done is extend characteristics of problems and solutions, extend list of validation strategies, flesh out descriptions, show connections between problem/solution lists and validation, and provide citations for good examples>>>>
This page is part of Mary Shaw's site in the School of Computer Science at Carnegie Mellon University. It was last modified on 11/21/97. Use of any portion of this site to generate spam or other mass communications is forbidden. Comments to maintainer.