Newsgroups: comp.ai.neural-nets
Path: cantaloupe.srv.cs.cmu.edu!bb3.andrew.cmu.edu!newsfeed.pitt.edu!godot.cc.duq.edu!newsgate.duke.edu!news.mathworks.com!newsfeed.internetmci.com!realtime.net!news.mindspring.com!snooze.ser.bbnplanet.com!es.dupont.com!esds01.es.dupont.com!slivova.es.dupont.com!owens
From: owens@slivova.es.dupont.com (Aaron J. Owens)
Subject: Re: 2 layer nets and non-linearity
Message-ID: <1996May21.142021.21020@es.dupont.com>
Sender: news@es.dupont.com (USENET News System)
Nntp-Posting-Host: slivova.es.dupont.com
Organization: DuPont Experimental Station
X-Newsreader: TIN [version 1.2 PL0]
References: <4nmqbg$77h@otis.netspace.net.au>
Date: Tue, 21 May 1996 14:20:21 GMT
Lines: 37

luke (luke@ice.net.au) wrote:
: Can a 2 layer net extract non-linear relationships from the data?

: I am researching time series prediction, and have found that
: performance is better (in terms of prediction accuracy and the
: correlation coefficient between the target/net output)  when a 2 layer
: net is used as opposed to using hidden layer(s).

: I have found 2 layer nets to be better over a variety of problems,
: from forecasting 5 minute moves in futures markets, to quarterly stock
: market returns.

I assume that by a "2 layer net" that you mean a network with input 
and output layers but no hidden layers (i.e., with one layer of weights).
Aside from output scaling by the squashing function, such a network
performs only a LINEAR mapping from inputs to outputs, essentially
equivalent to Multiple Linear Regression (MLR).

In many circumstances, especially forecasting in capital markets, the
nature of the data limits the order of the appropriate model to linear
or sub-linear (i.e., fewer active parameters than in MLR). Highly non-
linear models, as developed with a hidden layer and many hidden units,
tend to over-fit the training data unless very early stopping is applied.

-- Aaron --

Aaron J. Owens, Research Fellow    Telephone Numbers:
Modeling and Simulation		  	Office	  (302) 695-7341 
Engineering Research Laboratory	  	FAX	    "   695-9658
DuPont Company, E320/201	        Home      (302) 733-7836
Wilmington, DE 19880-0320	   Internet: owens@prism.es.dupont.com

-----------------------------------------------------------------------------
Opinions expressed in this electronic message should *NOT* be taken to repre-
sent the official view(s) of the DuPont Company.  ANY OPINIONS EXPRESSED ARE
THE PERSONAL VIEWS OF THE AUTHOR ONLY. 
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
