Newsgroups: comp.ai.neural-nets
Path: cantaloupe.srv.cs.cmu.edu!das-news2.harvard.edu!news2.near.net!howland.reston.ans.net!gatech!concert!sas!mozart.unx.sas.com!saswss
From: saswss@hotellng.unx.sas.com (Warren Sarle)
Subject: Re: NeuroGenetic Optimizer
Originator: saswss@hotellng.unx.sas.com
Sender: news@unx.sas.com (Noter of Newsworthy Events)
Message-ID: <D0EKqn.H4z@unx.sas.com>
Date: Tue, 6 Dec 1994 18:59:10 GMT
References:  <3bmbc9$d20@ixnews1.ix.netcom.com>
Nntp-Posting-Host: hotellng.unx.sas.com
Organization: SAS Institute Inc.
Lines: 35


In article <3bmbc9$d20@ixnews1.ix.netcom.com>, francone@ix.netcom.com (Frank Francone) writes:
|> I recently got literature on an interesting Windows software package
|> called NeuroGenetic Optimizer. It claims to select the relevant
|> independent variables from many possible independent variables and to do
|> the same with backprop network architectures. The details in the
|> literature I got are very sparse.

I know nothing about this product. However, selecting a subset of inputs
is very difficult to do well even in the case of a linear model.
Regularization methods such as ridge regression (weight decay) or
James-Stein estimation usually work better.  For a nonlinear model such
as an MLP, and especially for nonlinear models that are often highly
overparameterized by the usual statistical standards, subset selection
is likely to be even more difficult. Since you would have a complicated
combinatorial search, covering both inputs and numbers of hidden units,
a genetic algorithm might well be a good thing to try, but I would
suggest that you be wary of the results and carefully validate things
yourself. Does the literature say anything about how the product deals
with the issue of getting good generalization?

Reference for subset selection in linear models:

  Miller, A.J. (1990), Subset Selection in Regression, Chapman & Hall.

Reference comparing subset selection with other popular stuff:

   Frank, I.E. & Friedman, J.H. (1993) "A statistical view of some
   chemometrics regression tools," Technometrics, 35, 109-148.

-- 

Warren S. Sarle       SAS Institute Inc.   The opinions expressed here
saswss@unx.sas.com    SAS Campus Drive     are mine and not necessarily
(919) 677-8000        Cary, NC 27513, USA  those of SAS Institute.
