Newsgroups: comp.ai
Path: cantaloupe.srv.cs.cmu.edu!europa.chnt.gtegsc.com!gatech!howland.reston.ans.net!tank.news.pipex.net!pipex!uknet!newsfeed.ed.ac.uk!dcs.ed.ac.uk!calvay.dcs.ed.ac.uk!mjh
From: mjh@dcs.ed.ac.uk (Mark Hartswood)
Subject: The success of (medical) decision support systems
Message-ID: <DCzswt.5so@dcs.ed.ac.uk>
Sender: mjh@calvay.dcs.ed.ac.uk (Mark Hartswood)
Organization: Department of Computer Science, University of Edinburgh
X-Newsreader: xrn 7.03
Date: Tue, 8 Aug 1995 12:48:29 GMT
Lines: 92

Hello,

I am involved with a project to develop a system which generates
prompts for screening radiologists based upon algorithmic interpretation
of mammograms.

My research is towards a PhD, and is largely based around Human 
Factors considerations in the design and delivery of this technology.

A literature review has revealed that although many medical decision 
support systems have been built (1000's), few are in routine operation 
(10's) -- and many of those in only a single location.

There are conflicting views as to why this might be the case, many
authors state that there have been methodological problems (eg [1])
with knowledge acquisition, design and implementation, while others
have stated that systems simply do not provide an acceptable level
of accuracy (eg [2]).

My interest in this phenomena is two-fold: firstly I am keen to ensure
that the project of which I am a part does not fall foul of pit falls others
have experienced. Secondly, I have an academic interest in investigating
further the reasons for the perceived lack of success of such systems. For
example, it might be interesting to ascertain whether this 
phenomena is unique to the medical context - or whether there is a
problem with routine use of decision support systems in other contexts. 

To these ends I would be very interested to make contact both developers
and users of decision support systems in any application context 
(especially - if the context is a medical one) with the aim of gathering 
case histories, anecdotes, opinions and pointers to other information 
sources. It is often difficult to respond to such a general request
for information, so I have included some specific questions below. Feel
free to give further information, or ignore particular questions as
appropriate.

  What is the application context of your system?

  Is your system in routine use, or is it still in development?

    If it is in routine use:
      How long has it been in routine use?
      How many sites is it used in?
      Was there any resistance to its introduction?
      Is system usage at an expected level?

  What indicated that the use of decision support systems would
  be valuable in this context?

  Have you performed any human-factors work yourself, either with respect
  to knowledge acquisition, system design, or evaluation? If, so have you
  found such studies useful?

  How did you determine the criteria for evaluating the systems
  performance?

  Have you found any differences between the performance indicated by
  laboratory testing and performance in practice? 

  How do users respond to your system in practice?
  (Do they find it helpful/unhelpful? Do they actually use it? Are
  there any problems associated with interpreting it's output? What
  false negative / false positive rates do they find acceptable? What
  are their attitudes towards it generally (Cautious? Optimistic?) ?)

  (This is a difficult one) How do the users think the system
  will alter the nature of the job they perform? (Do they worry about
  de-skilling? Do they think they think it will enable them to function
  more effectively?)


I do read this group regularly, but would be grateful for e-mail
responses. If permission is given I will summarise and post any
substantive replies. If there is a large response I will consider
setting up a web resource concerning these issues.

Many thanks,

Mark Hartswood
mjh@dcs.ed.ac.uk


[1] H. A. Heathfield, J. Wyatt (1993)
    "Philosophies for the design and development of clinical decision
     support systems"
    Methods of Information in Medicine.
    32; 1-8

[2] G. C. Sutton (1989)
    "Computer-aided diagnosis: a review."
    British Journal of Surgery, vol78 pp82-85

