Newsgroups: comp.ai.neural-nets
Path: cantaloupe.srv.cs.cmu.edu!bb3.andrew.cmu.edu!newsfeed.pitt.edu!godot.cc.duq.edu!news.duke.edu!news.mathworks.com!newsfeed.internetmci.com!in2.uu.net!news.interpath.net!sas!newshost.unx.sas.com!saswss
From: saswss@hotellng.unx.sas.com (Warren Sarle)
Subject: Re: Testing an ART network
Originator: saswss@hotellng.unx.sas.com
Sender: news@unx.sas.com (Noter of Newsworthy Events)
Message-ID: <DrB63s.9tA@unx.sas.com>
Date: Sun, 12 May 1996 19:57:28 GMT
X-Nntp-Posting-Host: hotellng.unx.sas.com
References: <4muep4$ksi@baggins.cc.flinders.edu.au> <4mvlmj$nb7@bignews.shef.ac.uk>
Organization: SAS Institute Inc.
Lines: 41


In article <4mvlmj$nb7@bignews.shef.ac.uk>, COP95AJW@shef.ac.uk (COP95AJW) writes:
|> In article <4muep4$ksi@baggins.cc.flinders.edu.au>, 
|> kinslow@ist.flinders.edu.au says...
|>   All the books I've read on ART networks are based around training the net.
|> >How do you test it ?  It would seem rather pointless if you couldn't extract
|> >information from the net.
|> >        Any responses (including smart ass comments) are welcome.
|> 
|> Which type of ART network are you using? If its a supervised version (ARTMAP, 
|> fuzzy ARTMAP) then you can test it as you would an MLP - ie presenting test 
|> inputs and comparing the resulting outputs with those desired. If its 
|> unsupervised then things are a bit different. Since there is generally no 
|> correct categerisation structure that should be learned then you cannot test 
|> wether or not a correct one has been learnt. 

You seem to be saying that there is no "right answer" for unsupervised
ART networks. But if there is no way to tell (even in theory if not in
practice) whether a result is correct or not, the method is worthless.

There are various models by which clustering methods can be evaluated.
For example, see:

   Balakrishnan, P.V., Cooper, M.C., Jacob, V.S., and Lewis, P.A. (1994)
   "A study of the classification capabilities of neural networks using
   unsupervised learning: A comparison with k-means clustering",
   Psychometrika, 59, 509-525.

Even if for most data sets the "correct" results are not clearly
defined, one would hope it would be possible to concoct some artificial
situation so perfectly clear that the "correct" answer would be
indisputable. I tested Fuzzy ART on a few such cases and found that the
results depended almost entirely on the order in which the training
cases were presented and not on the distribution of the training cases.
See ftp://ftp.sas.com/pub/neural/fart.doc .

-- 

Warren S. Sarle       SAS Institute Inc.   The opinions expressed here
saswss@unx.sas.com    SAS Campus Drive     are mine and not necessarily
(919) 677-8000        Cary, NC 27513, USA  those of SAS Institute.
