Newsgroups: comp.ai.neural-nets
Path: cantaloupe.srv.cs.cmu.edu!bb3.andrew.cmu.edu!nntp.sei.cmu.edu!news.cis.ohio-state.edu!news.maxwell.syr.edu!news.mathworks.com!newsgate.duke.edu!interpath!news.interpath.net!news.interpath.net!sas!newshost.unx.sas.com!saswss
From: saswss@hotellng.unx.sas.com (Warren Sarle)
Subject: Re: new results on 1 vs. 2 hidden layers
Originator: saswss@hotellng.unx.sas.com
Sender: news@unx.sas.com (Noter of Newsworthy Events)
Message-ID: <E726rx.4zs@unx.sas.com>
Date: Sat, 15 Mar 1997 00:31:09 GMT
X-Nntp-Posting-Host: hotellng.unx.sas.com
References:  <5g9jca$juh$1@Jupiter.Mcs.Net>
Organization: SAS Institute Inc.
Lines: 35


In article <5g9jca$juh$1@Jupiter.Mcs.Net>, drt@MCS.COM (Donald Tveter) writes:
|> Everyone always wants to know how many hidden layers and how many
|> units to use.  There was that result that given N patterns you can
|> approximate most normal functions using N-1 hidden layer units in
|> a three layer network (lots of fine print).

You can obviously fit N cases exactly with N-1 hidden units as
long as no two cases have the same inputs but different targets.

|> Now in the March IEEE
|> Trans. on NN there is an article that proves the same can be done
|> with two hidden layers using only N/2+3 hidden layer units.  This
|> is interesting because the fewer weights you use the better the
|> results will be.  

In practice, subject to the same condition as above, you can virtually
always fit N cases exactly with one hidden layer with N/2 units. I would
be much more interested to see a counterexample for this situation.

|> Note that this paper is only a proof, there are
|> no experimental results.  As with the previous three layer network
|> proof there is no proof that backprop will find such good networks
|> only that the networks exist.

For one hidden layer with N units, backprop _will_ always eventually
get a perfect fit unless you set the learning rate too high.

-- 

Warren S. Sarle       SAS Institute Inc.   The opinions expressed here
saswss@unx.sas.com    SAS Campus Drive     are mine and not necessarily
(919) 677-8000        Cary, NC 27513, USA  those of SAS Institute.
 *** Do not send me unsolicited commercial or political email! ***

