Newsgroups: comp.ai.neural-nets
Path: cantaloupe.srv.cs.cmu.edu!bb3.andrew.cmu.edu!newsfeed.pitt.edu!gatech!newsfeed.internetmci.com!tank.news.pipex.net!pipex!lade.news.pipex.net!pipex!bt!btnet!zetnet.co.uk!demon!sunsite.doc.ic.ac.uk!yama.mcc.ac.uk!thor.cf.ac.uk!scmcms
From: C.M.Sully@cs.cf.ac.uk (Chris Sully)
Subject: Re: Multiple Hidden Layers - Why?
Sender: news@cf.ac.uk (USENET News System)
Message-ID: <DLC4BA.4zo@cf.ac.uk>
Date: Wed, 17 Jan 1996 16:54:46 GMT
X-Nntp-Posting-Host: aquamarine.cs.cf.ac.uk
References: <4d8j1c$og@percy.cs.bham.ac.uk> <4ddkmj$orj@ahorn.she.de> <4dhsl6$40v@delphi.cs.ucla.edu> <4diufc$kuh@willow.cc.kcl.ac.uk>
Organization: Dept of Computer Science, Univ of Wales, CARDIFF
Lines: 49

In article <4diufc$kuh@willow.cc.kcl.ac.uk>, Bharat Purohit <bow> writes:
|> -- 
|> Hi,
|> 
|> 
|> The original question was why you need multiple hidden layers in a network like the
|> MLP.
|> First of all if you look for the answer to this in books you might see apparently
|> contradictory information. Some books might say that you need three layeres, others
|> might say you need two and still otheres might say you need one!
|> 
|> Which one is right?
|> 
|> Well the problem is that some  books refer to a layer as a set of "neurons". Others
|> refer to a layer as a set of weights. So if your network had a layer of input
|> neurons, a layer of hidden neurons and a layer of output neurons, you could call
|> this a 3 layer (of neurons) network.
|> 
|> This is all you need to approximate any function (to get any input-output mapping).
|> However you may need to use a huge number of neurons wihtin the hidden layer.
|> Generally the more you use, the better the approximation.

Depending on the complexity of the function you are modelling. Increasing
the number of neurons increases the range of functions the network can represent. Thus if the complexity of the relationship between inputs and outputs
in your data is unknown you should start with a large number of HUs to give the
net the best chance of being able to model this function.

I'd go for 2-layers of adaptive weights for a description of the standard MLP.
More meaningful and less confusing.

Cheers.

Chris.

'I used to think that *I* was stupid, and then I met philosophers.'
        -- (Terry Pratchett, Small Gods)

========================================================
Christopher Sully
Ph.D. Student (Neural Networks)  
Department of Computer Science (Room C2.06), 
University of Wales College of Cardiff,   
Newport Road, CARDIFF, CF2 3XF. (Wales, UK) 
E-Mail: C.M.Sully@cs.cf.ac.uk
WWW: http://www.cs.cf.ac.uk/User/C.M.Sully
Phone:  +44 (0)1222 874000 x6068
Fax:	+44 (0)1222 666182
Home:	+44 (0)1222 484494
=========================================================
