Newsgroups: comp.ai.neural-nets
Path: cantaloupe.srv.cs.cmu.edu!rochester!udel!gatech!news.mathworks.com!tank.news.pipex.net!pipex!news.sprintlink.net!redstone.interpath.net!sas!mozart.unx.sas.com!saswss
From: saswss@hotellng.unx.sas.com (Warren Sarle)
Subject: Re: SuperSAB -- Pattern vs. epoch
Originator: saswss@hotellng.unx.sas.com
Sender: news@unx.sas.com (Noter of Newsworthy Events)
Message-ID: <DFHA6H.7H1@unx.sas.com>
Date: Mon, 25 Sep 1995 20:27:53 GMT
X-Nntp-Posting-Host: hotellng.unx.sas.com
References:  <435vpu$6t3@nz12.rz.uni-karlsruhe.de>
Organization: SAS Institute Inc.
Lines: 41


In article <435vpu$6t3@nz12.rz.uni-karlsruhe.de>,
riedml@i11s11.ira.uka.de (Martin Riedmiller) writes:
|> ...
|> In our experience, SuperSAB is faster and more roboust than
|> ordinary gradient descent with a fixed learning rate. However, the
|> computed weight update still depends on the size of the gradient.
|> That means, that even a carefully adapted learning rate may not
|> always prevent unfavourable weight updates, due to the unforeseeable
|> influence of the size of the partial derivative.

Martin is being very diplomatic. The truth of the matter is that
SuperSAB is grossly unstable. Rprop blows it away. Rprop is more
reliable, more efficient, and more elegant. There is absolutely
no reason ever to use SuperSAB if you can use Rprop (or Quickprop)
instead.

|> A comparison of SuperSAB, Backprop, Quickprop (Fahlman 1988) and
|> Rprop (Riedmiller and Braun, 1993) can be found in the following
|> articles (available at our ftp-server; see below):

ftp://i11s16.ira.uka.de/pub/neuro/papers

|> Martin Riedmiller and Heinrich Braun (1993) (riedml.icnn93.ps.Z):
|> A Direct Adaptive Method for Faster Backpropagation Learning:
|> The RPROP Algorithm,
|> Proceedings of the IEEE International Conference on Neural Networks
|> (ICNN) 1993,
|> San Francisco, CA, 1993
|>
|> Martin Riedmiller (1994) (riedml.csi94.ps.Z)
|> Advanced Supervised Learning in Multi-layer Perceptrons -
|> From Backpropagation to Adaptive Learning Algorithms
|> "Computer Standards & Interfaces", volume 16,
|> special issue on neural networks, edited by J. Fulcher,
|> Elsevier Science Publishers, Amsterdam, 1994.
-- 

Warren S. Sarle       SAS Institute Inc.   The opinions expressed here
saswss@unx.sas.com    SAS Campus Drive     are mine and not necessarily
(919) 677-8000        Cary, NC 27513, USA  those of SAS Institute.
