Newsgroups: comp.ai.neural-nets
Path: cantaloupe.srv.cs.cmu.edu!bb3.andrew.cmu.edu!newsfeed.pitt.edu!gatech!csulb.edu!hammer.uoregon.edu!hunter.premier.net!www.nntp.primenet.com!nntp.primenet.com!dispatch.news.demon.net!demon!usenet2.news.uk.psi.net!uknet!usenet1.news.uk.psi.net!uknet!uknet!newsfeed.ed.ac.uk!dcs.ed.ac.uk!ramna.dcs.ed.ac.uk!iic
From: Ian Clarke <iic@dcs.ed.ac.uk>
Subject: Re: Q: Backpropagation
In-Reply-To: <32709645.3E58@ish.dtu.dk>
X-Nntp-Posting-Host: ramna.dcs.ed.ac.uk
Content-Type: TEXT/PLAIN; charset=US-ASCII
Message-ID: <Pine.SOL.3.94.961031123705.12039C-100000@ramna.dcs.ed.ac.uk>
To: Lars Husted <ishlh@ish.dtu.dk>
Sender: cnews@dcs.ed.ac.uk (UseNet News Admin)
Organization: Department of Computer Science, Edinburgh University
References: <32709645.3E58@ish.dtu.dk>
Mime-Version: 1.0
Date: Thu, 31 Oct 1996 12:47:21 GMT
Lines: 63


I think I can help you with some of your questions

On Fri, 25 Oct 1996, Lars Husted wrote:
> 
> 1) Why must the input to the input layer be between 0 and 1 (why doesn't the weights compensate for this)?

Strictly speaking it depends upon the function you are using to map the
input of a neuron, to it's output, most functions try to have some kind of
curve, but this curve is only useful for values 0-1, on a larger scale it
strainghtens out.
 
> 4) Is it better to split the samples into two groups: one for training
>    and one for validation (why not validate on all the samples instead
>    of just some)?

The reason people seporate the sample data into a training, and a
validation group is so that they may test to see how well the neural
network has generalised.  For example, if I told you that

1,2 --> 3
7,9 --> 16
2,5 --> 7

(the unknown function being x,y --> x+y)

Surely, rather than asking:

7,9 --> ??

It would be more telling if I gave you some inputs which were not in the
training data.

> 5) How do I determine when to stop training?
>    As mentioned above I want the error to bee as low as possible, but
>    during training the error sometimes increases a bit before it
>    suddenly decreases a lot (this means that I can't stop training as
>    soon as the error starts to increase). 

Generally you should really either specify the maximum error you can
tolerate, and wait until the error drops below this, or specify the
maximum time that you can afford to wait.  You could also have an
algorithm that says:

if the error now is less than xxx% lower than what it was yyy training
cycles ago, then stop training.

In any of these circumstances, you must still come up with what may simply
be arbitrary figures, such is life.

Ian

/-------------------------------------------------------------------\
|IAN CLARKE | Preferred Email:    | "I am the thought you are       |
|----------/   iic@dcs.ed.ac.uk   |    thinking"                    |
|-------------------------------------------------------------------|
|  World Wide Web>> http://www.geocities.com/CapeCanaveral/2108/    |
|-------------------------------------------------------------------|
|  Student of Artificial Intelligence and Computer Science at the   |
|               University of Edinburgh, Scotland.                  |
\___________________________________________________________________/


