Newsgroups: comp.ai.neural-nets
Path: cantaloupe.srv.cs.cmu.edu!europa.chnt.gtegsc.com!howland.reston.ans.net!news.sprintlink.net!redstone.interpath.net!sas!mozart.unx.sas.com!saswss
From: saswss@hotellng.unx.sas.com (Warren Sarle)
Subject: Re: HELP: WHY NOT CONVERGING ?
Originator: saswss@hotellng.unx.sas.com
Sender: news@unx.sas.com (Noter of Newsworthy Events)
Message-ID: <D8J54E.st@unx.sas.com>
Date: Sat, 13 May 1995 18:38:38 GMT
X-Nntp-Posting-Host: hotellng.unx.sas.com
References:  <3ovs4h$epj@ictpsp10.ictp.trieste.it>
Organization: SAS Institute Inc.
Keywords: ANN linear identification
Lines: 36


In article <3ovs4h$epj@ictpsp10.ictp.trieste.it>, lonzam@elettra.trieste.it (Marco Lonza) writes:
|> I am trying to use ANN techniques to linearly identify systems, which means
|> find a linear representation (i.e. a matrix) of a black box with multiple inputs
|> and outputs (82 in and 96 out). The NN is a very simple FeedForward net, with
|> linear activation function and no hidden layers.

I.e., a linear regression model.

|> As a first step I use a pattern set generated from a matrix (96 rows by 82
|> columns) with random inputs and also with simple Standard BP the net converges
|> rapidly to the values of the original matrix.
|> Troubles come when I try to identify the inverse system (96 inputs and 82
|> outputs) using the same pattern set. I expected to achieve the pseudo inverse
|> of the original matrix

The least squares solution for B in Y=XB is not a generalized inverse
of the least squares solution for C in X=YC. Consider the case where
X and Y have only one column and are standardized; B and C are not
inversely related at all--they're identical!

|> or one of the other invertion solutions, but the net
|> doesn't converge; with StdBP I am obliged to keep ETA very small (0.01)
|> in order to avoid oscillations but the error is far away from the minimum
|> and with other algorithms (quickprop or RPROP) the situation doesn't change.

Those symptoms indicate that input matrix is ill-conditioned. If you
use any good linear regression program, you can get the least-squares
solutions without any iterating or juggling with learning rates, and
you can get diagnostics for ill-conditioning in the bargain.

-- 

Warren S. Sarle       SAS Institute Inc.   The opinions expressed here
saswss@unx.sas.com    SAS Campus Drive     are mine and not necessarily
(919) 677-8000        Cary, NC 27513, USA  those of SAS Institute.
