Newsgroups: comp.ai.neural-nets
Path: cantaloupe.srv.cs.cmu.edu!rochester!cornell!travelers.mail.cornell.edu!news.kei.com!news.mathworks.com!uunet!cs.utexas.edu!howland.reston.ans.net!news.sprintlink.net!redstone.interpath.net!sas!mozart.unx.sas.com!saswss
From: saswss@hotellng.unx.sas.com (Warren Sarle)
Subject: Re: On computing # nodes in hidden layer...
Originator: saswss@hotellng.unx.sas.com
Sender: news@unx.sas.com (Noter of Newsworthy Events)
Message-ID: <D4Itr0.L7B@unx.sas.com>
Date: Fri, 24 Feb 1995 20:18:36 GMT
X-Nntp-Posting-Host: hotellng.unx.sas.com
References:  <3iisj5$jbc@wizard.uark.edu>
Organization: SAS Institute Inc.
Keywords: ANN Hidden layer
Lines: 52


In article <3iisj5$jbc@wizard.uark.edu>, arb2@engr.uark.edu writes:
|> ...
|> Input=M
|> Hidden 1=N1= Sqrt(M*N2)
|> Hidden 2=N2= Sqrt(N1*P)
|> Output=P
|> ...
|> I know the whole square root thing is just a guideline and a lot of this is
|> guess work, but what are the guidelines for two hidden layers?

Perhaps my previous reply was not clear enough. The "square root thing"
is not a "guideline" or "guess work". It is crap. As I said before:

| All rules of thumb based on the numbers of inputs and outputs are utter,
| unredeemable nonsense. A net with 1 input and 1 output may need
| 1,000,000 hidden nodes. A net with 1,000 inputs and 1,000 outputs may
| need 1 (or zero) hidden nodes.  The optimal number of hidden nodes
| depends on:
| 
|  * The number of training cases.
| 
|  * The amount of noise.
| 
|  * The desired accuracy of generalization.
| 
|  * The training method.
| 
|  * The type of activation functions.
| 
|  * Details of the function being learned, which are difficult to describe
|    and which you probably don't know anyway.
| 
|  * Fill in the blank ____________________________________________
| 
| A discussion of practical methods for choosing the number of hidden
| nodes would run dozens to hundreds of pages, but to summarize:
| 
|  * With stopped training or regularization, use _lots_ of hidden
|    nodes.
| 
|  * Otherwise, start with 1 hidden node and work your way up, using
|    some estimate of generalization error to make the decision.
| 
| Either way, it's also a good idea to consider direct connections from
| inputs to outputs.

-- 

Warren S. Sarle       SAS Institute Inc.   The opinions expressed here
saswss@unx.sas.com    SAS Campus Drive     are mine and not necessarily
(919) 677-8000        Cary, NC 27513, USA  those of SAS Institute.
