Newsgroups: comp.ai.neural-nets
Path: cantaloupe.srv.cs.cmu.edu!rochester!cornellcs!travelers.mail.cornell.edu!news.kei.com!simtel!news.sprintlink.net!redstone.interpath.net!sas!mozart.unx.sas.com!saswss
From: saswss@hotellng.unx.sas.com (Warren Sarle)
Subject: Re: Implementation of LM nnet training algorithm in MATLAB
Originator: saswss@hotellng.unx.sas.com
Sender: news@unx.sas.com (Noter of Newsworthy Events)
Message-ID: <DF30Aw.Ep1@unx.sas.com>
Date: Mon, 18 Sep 1995 03:28:08 GMT
X-Nntp-Posting-Host: hotellng.unx.sas.com
References:  <436onm$rmq@taco.cc.ncsu.edu>
Organization: SAS Institute Inc.
Keywords: LM, MATLAB, Levenberg, Marquardt
Lines: 54


This question seems to be approaching FAQ status.

In article <43dclg$1eju@ds2.acs.ucalgary.ca>, "Khaled Ali M. El-Metwally" <metwally> writes:
|>     I'm wondering if some body can guid me to a refernce to the Levenberg-
|> Marquardt  Training method ??

   Fletcher, R. (1987) Practical Methods of Optimization, Wiley: NY.

   Levenberg, K. (1944) "A method for the solution of certain problems
   in least squares," Quart. Appl. Math., 2, 164-168.

   Marquardt, D. (1963) "An algorithm for least-squares estimation of
   nonlinear parameters," SIAM J. Appl. Math., 11, 431-441.

   More\', J.J. (1977) "The Levenberg-Marquardt algorithm:
   implementation and theory," in Watson, G.A., ed., _Numerical
   Analysis_, Lecture Notes in Mathematics 630, Springer-Verlag,
   Heidelberg, 105-116.


And in article <436onm$rmq@taco.cc.ncsu.edu>, bgkerman@eos.ncsu.edu (Bahram G Kermani) writes:
|> I was wondering if someone could kindly give me some reference/information
|> about the implementation of the LM algorithm in the NNet toolbox of MATLAB.
|>
|> As  I understand, this algorithm has been developed by Professor Martin Hagan
|> of Oklahoma State University.

As one might guess from the name, the Levenberg-Marquardt algorithm
was developed by Levenberg and Marquardt. Perhaps Professor Martin
Hagan wrote the particular program in question.

|> The reason I am asking about the specific implementation is that there are
|> numerous modifications of this algorithm out there.

True.

|> It has been shown that the
|> LM algorithm in its original form is useless unless the input data points are
|> not correlated. 

Utter nonsense. Levenberg's original algorithm was rather slow, but no
version of LM has ever had such a restriction. In fact, some variants,
like the closely-related Gauss-Newton algorithm, are invariant under
affine transformations of the Jacobian. 

|> I appreciate it very much if someone could clarify this issue for me.
|> Please email directly, if you could.

-- 

Warren S. Sarle       SAS Institute Inc.   The opinions expressed here
saswss@unx.sas.com    SAS Campus Drive     are mine and not necessarily
(919) 677-8000        Cary, NC 27513, USA  those of SAS Institute.
