Newsgroups: comp.ai.neural-nets
Path: cantaloupe.srv.cs.cmu.edu!das-news2.harvard.edu!news2.near.net!howland.reston.ans.net!news.sprintlink.net!crash!pzcc.bitenet!news
From: Duane DeSieno <duaned@cts.com>
Subject: Re: question about the momentum in backprop net
Organization: /etc/organization
Date: Tue, 24 Jan 1995 04:06:58 GMT
Message-ID: <D2w63o.EBA@crash.cts.com>
References:  <D2u82A.4By@cc.umontreal.ca>
Sender: news@crash.cts.com (news subsystem)
Nntp-Posting-Host: loci.cts.com
Lines: 23

> 	Could somedy explain what is exactly the role of the momentum in
> 	a backpropagation neural network ?
> 	If not, is it possible to get some good reference ?
> 
> Thank you.   
> 
> 
The popular belief is that momentum will speed the convergence of the
network training, and that it is an effective means of escaping local
minima.  Momentum adds to the current gradient a certain fraction of 
the prior weight change, producing the new weight change.  This has
a "snow-ball" effect on the weight changes, effectively increasing
the learning rate.

Care must be taken to avoid a "stalling" of the weights, caused by
growing the weights too quickly.

Hope this helps.

Duane DeSieno
Logical Designs
(619)459-6236
duaned@cts.com
