Newsgroups: comp.ai.neural-nets
Path: cantaloupe.srv.cs.cmu.edu!bb3.andrew.cmu.edu!newsfeed.pitt.edu!gatech!newsfeed.internetmci.com!EU.net!peer-news.britain.eu.net!newsfeed.ed.ac.uk!dcs.ed.ac.uk!jura.dcs.ed.ac.uk!iic
From: iic@dcs.ed.ac.uk (Ian Clarke)
Subject: Re: Q: Implementing a neural network
X-Nntp-Posting-Host: jura.dcs.ed.ac.uk
Message-ID: <DKyqqu.G4u.0.staffin.dcs.ed.ac.uk@dcs.ed.ac.uk>
Sender: iic@jura.dcs.ed.ac.uk (Ian Clarke)
Organization: Department of Computer Science, University of Edinburgh
X-Newsreader: xrn 8.00
References:  <4cs7si$1fv@agate.berkeley.edu>
Date: Wed, 10 Jan 1996 11:32:51 GMT
Lines: 63

In article <4cs7si$1fv@agate.berkeley.edu>, dpetrou@ptarmigan.CS.Berkeley.EDU (David Petrou) writes:
>      Hello.  I'm working on implementing a recurrent neural network
> and have some questions.  In the case that there are many ways to do
> what I ask, I would prefer to do what more closely models biological
> neural networks.  This is my first stab, so please excuse any naive
> questions.
> 
>      In a recurrent neural network, what is the best way to traverse
> the nodes and update their state?  In other words, how should an
> ordering be chosen for visiting each node and weighing and adding its
> inputs, plugging this into the activation function, and generating an
> activation level?  My intuition is that the ordering will change the
> behavior of a recurrent net, as opposed to a feed forward net in which
> it seems that you can update the nodes in a particular layer in any
> arbitrary order, then move to the next layer, etc.

If your net is basically a feed-forward net with a few connections to
previous nodes then you should treat it like a normal net and just start
at the bottom and work upwards.  If not you might like to try randomly
updating the nodes and connections.

> 
>      Aside from modifying weights during learning/training of a neural
> net, are there situations in which nodes and/or connections are
> created or destroyed?  I can imagine the destruction of a connection
> resulting from a weight becoming zero, but are there instances in
> which new connections are created?

Not in any neural net scheme I have heard of.

> 
>      Also, during learing, does the activation function change?  Can a
> node use step at one point than move to sigmoid?  If so, what
> techniques are used to select the function?  Can the activation
> function be represented as something like a fourth degree polynomial
> with coefficients that are adjusted to simulate certain functions?

No, this would 'confuse' the training algorithm, nodes retain the same
activation function throughout.

> 
>      Is there any literature that I should read that would help me
> with implementing a neural net?  I have glanced at some books, but
> most of what I've found centers on specific learning algorithms, or
> the relation of neural nets to certain statistical models, or
> discussions on what different types of nets can represent, etc.  At
> this point, I would like to read a detailed explanation of how
> biological neural nets work with emphasis on building one in software.

"Introducing Neural Networks" By Alison Carling, Sigma Press, provides a
good over-view of several types of neural networks and training algorithms.

> 
> Thanks a lot,

No problem

Ian

-- 
|IAN CLARKE        I.Clarke@sms.ed.ac.uk "They couldn't hit an elephant|
|                  iic@dcs.ed.ac.uk      at this dist.."  - Last words |
|I.Clarke@ed.ac.uk ianc@aisb.ed.ac.uk    of General M Howard, 1918     |
