Newsgroups: comp.ai.neural-nets
Path: cantaloupe.srv.cs.cmu.edu!bb3.andrew.cmu.edu!nntp.sei.cmu.edu!toads.pgh.pa.us!investor.pgh.pa.us!rbp
From: rbp@investor.pgh.pa.us (Bob Peirce #305)
Subject: Re: On computing # nodes in hidden layer...
Message-ID: <1995Mar1.180342.5060@investor.pgh.pa.us>
Date: Wed, 1 Mar 95 18:03:42 GMT
Reply-To: rbp@investor.pgh.pa.us (Bob Peirce #305)
References: <3iir9f$j3m@wizard.uark.edu>
Organization: Cookson, Peirce & Co., Pittsburgh, PA
Keywords: ANN Hidden layer
Lines: 55

In article <3iir9f$j3m@wizard.uark.edu> arb2@engr.uark.edu writes:
>In article <Mark_Vogt.15.001147CE@qmgate.anl.gov>, Mark_Vogt@qmgate.anl.gov 
>(Mark Edward Vogt) writes:
>
>>Michael -
>>
>>A better model of your heuristic is:
>>
>>Input layer nodes = M
>>
>>Hidden layer nodes = N := Sqrt (M*P)
>>
>>Output layer nodes = P
>>
>>This allows no misconstruing of whether N = P or not.....
>>
>>Mark
>>
>>ps - It is my understanding that while a single hidden layer often suffices,
>>it is actually 2 (TWO) layers which guarantees approximation of any
>>function....
>
>That is what I understood as well.  So for two hidden layers, what should be 
>the number of neurons for the hidden layers:
>
>Hidden Layer 1=N1 := Sqrt (M*P)
>Hidden Layer 2=N2 := Sqrt (N1*P) ?
>
>I was trying to imagine a way to do this?  Any suguestions?
>What if you had only 8 inputs and 1 output (say I'm modeling a function)
>Would that be N1=3 and N1=2 AND P=1.

Maybe	N1 = (MN2)^0.5
	N2 = (N1P)^0.5  ??

I asked a question like this a while back and was told there is no
widely accepted theory on the ideal number of neurons to have in one or
more hidden layers, except most folks seem to think going much beyond
N = M risks memorizing instead of learning.

One source I read suggested the second hidden layer should be about
0.12M.  He also suggested that the first hidden layer should be
2(M+P)^0.5.  I have also tried M^0.5, which with P=1 becomes (MP)^0.5. 
I landed on that by accident using the assumption M/N ~= N/P.  I also
have tried N = M.   And, let's not forget connecting M to P as well as
N.  I am a novice, and nobody seems to have strong theories on why or
why not to do any of these things, so I just grope along and hope.  All
seem to produce somewhat similar, yet disturbingly different, results!
-- 
Bob Peirce                Pittsburgh, PA                 412-471-5320
rbp@investor.pgh.pa.us [OFFICE]    me@venetia.pgh.pa.us [HOME (NeXT)]

There is only one basic human right, the right to do as you damn well
please. And with it comes the only basic human duty, the duty to take
the consequences. -- P.J. O'Rourke
