Newsgroups: comp.ai.neural-nets
Path: cantaloupe.srv.cs.cmu.edu!das-news2.harvard.edu!news2.near.net!news.mathworks.com!news.alpha.net!uwm.edu!spool.mu.edu!howland.reston.ans.net!EU.net!uunet!olivea!news.bu.edu!gw1.att.com!princeton!handel!jtoth
From: jtoth@handel.Princeton.EDU (Gabor J.Toth)
Subject: Re: Why hidden layer (not VERY stupid Q)
Message-ID: <1994Oct25.150442.28388@Princeton.EDU>
Originator: news@hedgehog.Princeton.EDU
Sender: news@Princeton.EDU (USENET News System)
Nntp-Posting-Host: handel.princeton.edu
Organization: Princeton University
References: <v9110104-251094120955@igwemc25.vub.ac.be>
Date: Tue, 25 Oct 1994 15:04:42 GMT
X-Original-Newsgroups: comp.ai.neural-nets
Lines: 30

In article <v9110104-251094120955@igwemc25.vub.ac.be>,
Johan Ovlinger <v9110104@is2.vub.ac.be> wrote:
>Assuming a feedforwrd net trained in the standard back prop way:
>
>we can model an n layer net by matrix multiplication ( repr by :*):
>
>outputs  == An * A(n-1) * ... * A1 * inputs
>
>Linear algebra tells us that we can preform the multiplications in any
>order, so  it is equally valid to propagate the inputs through in the
>obviuos manner as it is to multiply the whole net into one composite matrix.

You are    perfectly right, and  this is   the reason nonlinear neural
networks were introduced. In a nonlinear multilayer network
  outputs ==  fn(An * f(n-1)(A(n-1) * ... * f1(A1*inputs))),
where f1,  ..., fn  are  some  *nonlinear* functions.  Because of  the
nonlinearity such networks can not be replaced by single layer ones.

Gabor

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
+++++++++++++++++++++++++++ Gabor J. Toth ++++++++++++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
++ Frick 316, Department of Chemistry ++++++++++ "All that glitters ++++
++ Princeton University ++++++++++++++++++++++++ may not be gold but +++
++ Princeton, NJ-08544 +++++++++++++++++++++++++ it certainly contains +
++ (609)254-6381 +++++++++++++++++++++++++++++++ free electrons." ++++++
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++


