Newsgroups: comp.ai.neural-nets
Path: cantaloupe.srv.cs.cmu.edu!das-news2.harvard.edu!news2.near.net!howland.reston.ans.net!gatech!concert!sas!mozart.unx.sas.com!saswss
From: saswss@hotellng.unx.sas.com (Warren Sarle)
Subject: Re: proof for backprop ??
Originator: saswss@hotellng.unx.sas.com
Sender: news@unx.sas.com (Noter of Newsworthy Events)
Message-ID: <D14KDI.91D@unx.sas.com>
Date: Tue, 20 Dec 1994 19:48:53 GMT
References: <1994Dec12.122304.44261@yogi> <3d4s37$48n@aplcomm.jhuapl.edu> <3d4vsr$t03@spool.cs.wisc.edu>
Nntp-Posting-Host: hotellng.unx.sas.com
Organization: SAS Institute Inc.
Lines: 37


In article <3d4vsr$t03@spool.cs.wisc.edu>, craven@cs.wisc.edu (Mark Craven) writes:
|> In article <3d4s37$48n@aplcomm.jhuapl.edu>, randy@aplcorejhuapl.edu (Randall C. Poe) writes:
|> |> In article <1994Dec12.122304.44261@yogi>, leimgruber1@urz.unibas.ch writes:
|> |> |> I'm looking for papers about the proof of the error backpropagation technic.
|> |> |> ...
|> |> The only proof I have ever seen is from Mangasarian in the Operations Research
|> |> literature.  Sorry, I don't have the reference, but it was something like
|> |> the Journal of OR, around '92 or '93.  The proof is under certain restrictions
|> |> on step sizes.
|>
|> This work was also presented at NIPS-6.  Here's the reference:
|>
|> @incollection{mangasarian.nips6
|>   ,author       = "O. L. Mangasarian and M. V. Solodov"
|>   ,title        = "Backpropagation convergence via deterministic perturbed minimization"
|>   ,booktitle    = "Advances in Neural Information Processing Systems"
|>   ,editor       = "J. Cowan and G. Tesauro and J. Alspector"
|>   ,year         = 1994
|>   ,volume       = 6
|>   ,pages        = "383--390"
|>   ,publisher    = "Morgan Kaufmann"
|>   ,address      = "San Francisco, CA"

As Randall C. Poe mentioned, this proof requires that the learning
rate and momentum go to zero at appropriate rates, so the proof does
_not_ apply to standard backprop. What is proved is convergence to a
stationary point; whether this is what the original poster was asking
for is not clear.



-- 

Warren S. Sarle       SAS Institute Inc.   The opinions expressed here
saswss@unx.sas.com    SAS Campus Drive     are mine and not necessarily
(919) 677-8000        Cary, NC 27513, USA  those of SAS Institute.
