Newsgroups: comp.ai.neural-nets
Path: cantaloupe.srv.cs.cmu.edu!bb3.andrew.cmu.edu!newsfeed.pitt.edu!scramble.lm.com!news.math.psu.edu!chi-news.cic.net!nntp.coast.net!howland.reston.ans.net!newsfeed.internetmci.com!portal.gmu.edu!hearst.acc.Virginia.EDU!news-server.ncren.net!redstone.interpath.net!sas!mozart.unx.sas.com!saswss
From: saswss@hotellng.unx.sas.com (Warren Sarle)
Subject: Re: What IS a neural-net?
Originator: saswss@hotellng.unx.sas.com
Sender: news@unx.sas.com (Noter of Newsworthy Events)
Message-ID: <DMvrIJ.Iuo@unx.sas.com>
Date: Fri, 16 Feb 1996 18:03:55 GMT
X-Nntp-Posting-Host: hotellng.unx.sas.com
References: <4env0k$i37@gensva.athena.livjm.ac.uk> <4eotuf$5lg@delphi.cs.ucla.edu>
Organization: SAS Institute Inc.
Lines: 91


I was hoping to see some responses to this article. If you have anything
to say, please chime in! 

In article <4eotuf$5lg@delphi.cs.ucla.edu>, edwin@cs.ucla.edu (E. Robert Tisdale) writes:
|> ...
|> Warren S. Sarle attempts to answer this question in

I didn't actually _write_ that answer. I presume Lutz did. But it is
now my responsibility to update the answer if people think that is
desirable.

|>      `comp.ai.neural-nets_FAQ,_Part_1_of_7:_Introduction'
|> under
|>      `What is a neural network (NN)?'
|> 
|> He does a fair job of answering the question
|> 
|>      `What do most people mean think of when they say "neural network"?'
|> 
|> but most people have only very vague notions of what constitutes
|> a neural network.  They tend to describe them in terms of unrelated
|> ideas that have been associated with them in the current literature.
|> This is usually sufficient for casual conversation but seldom adequate
|> for experts to determine the accuracy of any statement made about
|> neural networks or even whether or not they disagree with each other.
|> A clear definition of neural networks is required.
|> 
|> A neural network is a computational network
|> with the following three properties:
|> 1.)  simple processing units,
|> 2.)  massive parallelism and
|> 3.)  high connectivity.

I have come across similar definitions in several textbooks, but this
definition is not satisfactory from my point of view as a statistician.
I find it very useful to regard a wide variety of statistical models,
ranging from simple linear regression to generalized linear models and
other more exotic beasts, as special cases of neural networks. The
requirements of "massive parallelism" and "high connectivity" clearly
rule out anything as simple as simple linear regression.

|> For good reason, some features commonly associated with neural networks
|> are conspicuously absent from this definition.
|> 
|> 1.)  It does not imply that artificial neural networks which are the
|>      inventions of engineers are in any way derived from or even inspired
|>      by natural neural networks ...

Really? I thought the entire neural network field was inspired by the
desire to imitate natural neural networks.

|>      ... which are the nervous systems of animals
|>      but applies equally to both artificial and natural neural networks.
|> 
|> 2.)  It does not mention learning.  Despite the fact that the bulk
|>      of the literature on neural networks concerns itself with learning,
|>      most neural networks do not learn.  The function of natural neural
|>      networks is almost entirely determined by nature during development
|>      and changes very little over the life span of the host organism.

This is a bit outside my current work, but I recall from my college
psych classes that virtual all animals, even single-celled ones, exhibit
some form of learning. This makes it difficult for me to understand the
claim that "most neural networks do not learn". Certainly most
artificial neural networks learn!

|>      One can only hope that engineers will eventually mature beyond their
|>      current fascination with machines that learn and turn their attention
|>      to exploiting the raw computational power artificial neural networks
|>      offer.

Why?

[Point 3 and 4 omitted, being noncontroversial]

|> 5.)  It does not specify whether numeric or symbolic, analog or digital,
|>      continuous or discrete signals propagate through the network.  
|>      Any combination of signaling methods is possible.

Now this directly contradicts the FAQ, which says that neural networks
operate via numeric, not symbolic, signals. Anybody else have any
opinions on this question?  Any examples of neural networks that operate
via non-numeric signals?


-- 

Warren S. Sarle       SAS Institute Inc.   The opinions expressed here
saswss@unx.sas.com    SAS Campus Drive     are mine and not necessarily
(919) 677-8000        Cary, NC 27513, USA  those of SAS Institute.
