Newsgroups: comp.ai.neural-nets
Path: cantaloupe.srv.cs.cmu.edu!bb3.andrew.cmu.edu!newsfeed.pitt.edu!gatech!usenet.eel.ufl.edu!news.mathworks.com!nntp.primenet.com!news.fibr.net!news.sprintlink.net!news-fw-6.sprintlink.net!interpath!news.interpath.net!sas!newshost.unx.sas.com!saswss
From: saswss@hotellng.unx.sas.com (Warren Sarle)
Subject: Re: Two layer nets and non-linearity
Originator: saswss@hotellng.unx.sas.com
Sender: news@unx.sas.com (Noter of Newsworthy Events)
Message-ID: <Dws0qp.C7D@unx.sas.com>
Date: Tue, 27 Aug 1996 03:08:49 GMT
X-Nntp-Posting-Host: hotellng.unx.sas.com
References:  <4vmgh4$ima@otis.netspace.net.au>
Organization: SAS Institute Inc.
Lines: 39


In article <4vmgh4$ima@otis.netspace.net.au>, luke@ice.net.au (luke) writes:
|> Can a 2 layer net extract non-linear relationships from the data?

I assume that "2 layer net" means no hidden layer, with the inputs
linearly combined at the output layer. This is basically a "generalized
linear model". Such a network can learn nonlinear relationships of the
same form as the output activation function. It can't learn general
nonlinear functions of the inputs.

|> I am researching time series prediction, and have that performance is
|> always better when no hidden layers are used.   The more hidden
|> layers/nodes I add, the worse the performance is (in terms of the
|> correlation coefficient between the target/output and prediction
|> accuracy).

My personal opinion on this matter, which I hope may be of some general
validity, is given in the Neural Network FAQ, part 3 of 7:
Generalization, at ftp://ftp.sas.com/pub/neural/FAQ3.html:

Subject: How many hidden layers should I use?

You may not need any hidden layers at all. Linear and generalized linear
models are useful in a wide variety of applications (McCullagh and
Nelder 1989). And even if the function you want to learn is mildly
nonlinear, you may get better generalization with a simple linear model
than with a complicated nonlinear model if there is too little data or
too much noise to estimate the nonlinearities accurately. 

[lots of stuff deleted]

   McCullagh, P. and Nelder, J.A. (1989) Generalized Linear Models,
   2nd ed., London: Chapman & Hall.

-- 

Warren S. Sarle       SAS Institute Inc.   The opinions expressed here
saswss@unx.sas.com    SAS Campus Drive     are mine and not necessarily
(919) 677-8000        Cary, NC 27513, USA  those of SAS Institute.
