Newsgroups: comp.ai.neural-nets
Path: cantaloupe.srv.cs.cmu.edu!rochester!cornell!travelers.mail.cornell.edu!news.kei.com!news.mathworks.com!uunet!psinntp!nntp.hk.super.net!news.ust.hk!shunkong
From: shunkong@cs.ust.hk (*-- Michael --*)
Subject: Re: Q. of BPNN
Message-ID: <1995Feb22.180227.26335@uxmail.ust.hk>
Sender: usenet@uxmail.ust.hk (usenet account)
Nntp-Posting-Host: cssu135.cs.ust.hk
Organization: The Hong Kong University of Science and Technology
References: <1995Feb22.125046.19680@uxmail.ust.hk>
Date: Wed, 22 Feb 1995 18:02:27 GMT
Lines: 95

In article <1995Feb22.125046.19680@uxmail.ust.hk>,
Wong Tsz Cheong <cs_leo@ug.cs.ust.hk> wrote:
>Hi,
>	As I am doning the project about face recognition in Neural Network,
>	I have encountered some problems:
>
>1) By some books, the weights are adjusted when the training set is trained
>   once and the average sum-squared error is larger than the required maximum
>   error.
>   However, I have found a example BPNN program had a different idea.
>   It is that the weights are adjusted when one pattern(in the training set)
>   is trained once and the sum-squared error of this pattern is larger than
>   maximum error.
>
>   Could someone tell me why it works and which one is better?(e.g. errors
>   decrease faster.)

	I seldom heard of the first approach. So, I think the latter is the
better one. It works because it modify its weights for each sample pattern by
	i) computing the difference between the desired and the actual output;
	ii) feeding back this error level by level to the inputs, changing the
	    connection weights in such a way as to modify them 'in porportion'
	    to their responsibility for the output error.

	The slow convergence of the network is mainly deal to the training 
algorithm (ie the learning function), and the training set itself. If the samples 
posses:  i) small intraclass invariance, and
	ii) large interclass separation;
then, the training time will be shorter; otherwise....:P !

>
>
>2) I have put the required faces patterns in the training set. However, it
>   is not enough.  If a face which is not in our database is input in NN,
>   it should tell me it is not.  So, I need to train some error patterns
>   (means the faces not in our database) or garbage. But I can't include a
>   large number of error patterns since it will increase the training set's
>   size and also the number of hidden nodes. If a few error patterns are
>   included, the percentage of incorrect recognition will be large.

	If the feature extraction algorithm can extract "useful" information
from the patterns, I think you'll be fine. As I said earlier, your f.e. alg.
should give 'large interclass separation' (can tell the difference between 
'garbage' and the 'faces' with different values numerically). If that's your
problem, I think you have to work on the f.e. algorithm.

>
>   Could someone suggest some guidelines of training set for me?
>

	Here are my 'guidelines' of training, hope it helps :

1) avoid memorization : the number of hidden nodes should never = 
			the number of input nodes;
			otherwise, the hidden nodes will become
			some "graandmother cells" and have no idea at all
			what to generalize for a test-input pattern.	

2) joyful noise :	add some random noise to your input data;
			Oddly, networks sometimes train faster with such data!
			This technique forces the network to generalize,
					=> a KEY GOAL !!	

3) tolerance:		we determine the tolerance level for errors, and any 
			little mistakes below that level are considered perfect
			responses. Only errors > the tolerance level are used
			to generate weight changes. 

4) momentum:		Keep up the momentum. It helps the net to slid down a hill...

5) hidden layer nodes:	sometimes there is no other way, we have to use a bigger
			middle layer! We can increase the middle layer by 10% if
			the training is taking much too long ... give it a shake!

6) good luck:		utmost important....you know. :)


"Remember, in training the BPNN, the odd ideas are more in one's favor."

:P
Michael.
-- 
                ^  ^
*************   . .   **************************
               ( @  )
                 u
                                    Michael
Shun-Kong, Michael WAI             /______/\Wai
email   : shunkong@cs.ust.hk       \______\/__/\
tel. no.: (852)-2358-8833           No big \__\/
                                           deal!

***********************************************


