Newsgroups: comp.ai.neural-nets
Path: cantaloupe.srv.cs.cmu.edu!bb3.andrew.cmu.edu!newsfeed.pitt.edu!godot.cc.duq.edu!newsgate.duke.edu!news.mathworks.com!newsfeed.internetmci.com!in1.uu.net!news.interpath.net!sas!newshost.unx.sas.com!saswss
From: saswss@hotellng.unx.sas.com (Warren Sarle)
Subject: Re: Number of hidden units
Originator: saswss@hotellng.unx.sas.com
Sender: news@unx.sas.com (Noter of Newsworthy Events)
Message-ID: <DsHtC8.1B7@unx.sas.com>
Date: Tue, 4 Jun 1996 20:38:32 GMT
X-Nntp-Posting-Host: hotellng.unx.sas.com
References: <4oes5k$a6o@highway.leidenuniv.nl> <4p023h$1ta@llnews.ll.mit.edu>
Organization: SAS Institute Inc.
Lines: 40


In article <4p023h$1ta@llnews.ll.mit.edu>, heath@ll.mit.edu (Greg Heath) writes:
|> <1996May28.203735.6613@es.dupont.com>
|> Organization: MIT/Lincoln Laboratory
|> Keywords: 
|> To: owens@slivova.es.dupont.com (Aaron J. Owens)
|> 
|> In article <1996May28.203735.6613@es.dupont.com>, owens@slivova.es.dupont.com 
|> (Aaron J. Owens) writes:
|> |> Stephan Raaijmakers (raaijmakers@rulxho.leidenuniv.nl) wrote:
|> |> : What is the theoretical interpretation of 
|> |> : a NxMxN net (N=number of inputs/outputs; M=number of hidden units),
|> |> : where M>N? E.g., 24x74x24. Is it still possible to say that such a net
|> |> : uses a distributed representation?
|> |> 
|> |> Yes, all of the M hidden units participate in modeling the input-
|> |> output relationship -- if there are enough data points to allow that.
|> |> 
|> |> : Does anyone know of specific types of problems which demand that M>N?
|> |> 
|> |> Any situation in which a highly nonlinear mapping from input space
|> |> to output space is required, there is little noise, and there are
|> |> lots of exemplars.
|> 
|> Why the constraint on noise?

In order to learn a highly nonlinear mapping, you need either low
noise or lots of cases or some combination thereof, preferable both.

BTW, what is meant by "highly" nonlinear depends on the type of
network, since a given mapping may be easy for an MLP but hard for
an RBF net, or vice versa.



-- 

Warren S. Sarle       SAS Institute Inc.   The opinions expressed here
saswss@unx.sas.com    SAS Campus Drive     are mine and not necessarily
(919) 677-8000        Cary, NC 27513, USA  those of SAS Institute.
