Newsgroups: comp.ai.neural-nets
Path: cantaloupe.srv.cs.cmu.edu!das-news2.harvard.edu!news2.near.net!news.mathworks.com!news.alpha.net!uwm.edu!cs.utexas.edu!howland.reston.ans.net!Germany.EU.net!EU.net!nova.puug.pt!news.rccn.net!master.di.fc.ul.pt!news.inesc.pt!animal.inescn.pt!bart.inescn.pt!jcardoso
From: jcardoso@bart.inescn.pt (Joao Cardoso)
Subject: Pruning, feature selection and Info. Theory
Message-ID: <D6sDLH.D9p@animal.inescn.pt>
Sender: news@animal.inescn.pt (USENET News System)
Nntp-Posting-Host: bart.inescn.pt
Organization: INESC-Porto, Portugal
Date: Sun, 9 Apr 1995 21:12:51 GMT
Lines: 55

Hi,

I am working in an application of neural networks to industry, in the
quality control area.

The NN has to classify produced parts according to a pre-established
quality criteria. Each produced part is characterized by some
time-varying signals.

Each of those time-varying signals is transformed to the frequency
domain by a FFT, resulting in 512 spectral components. Those spectral
components are the  candidate inputs (features) for the NN.

The problem is: there are too many features and too few patterns.
Just to give an idea, there are between 100/200 patterns and 1000 features!
The classification of the produced parts is expensive!

The solution is feature selection.

Selecting the features with algorithms external to the NN does not seems
wise to me, as the search objective function can't guess the possible
complex mappings that will be performed latter by the NN. Of course, if
feature selection algorithms such as Min-Max, Branch and Bound and
others (clues accepted...) give good results, than no problem exists.
However, if the features selected with such algorithms give bad results
with the NN, we can never know if there are other features not selected
that would perform better.

The obvious choice would be to perform the feature selection in the NN.
Some of the existing pruning algorithms require a fully trained NN,
which is impossible in this situation. Thus, the ideal would be to prune
(the input units at least) during the learning phase. Does any
learning/pruning algorithm exists who can do this? A coarse initial
pruning to reduce the dimensionality to a manageable one, followed by a
more refined pruning, all this while learning?

Any hints, either in the statistical feature selection area or in 
prune-while-learn algorithms or still on information theory applied to NN
will be welcome.

I will post the results, if any...

Thanks,



Joao Cardoso
INESC
R. Jose Falcao 110
4000 PORTO
PORTUGAL

e-mail: jcardoso@bart.inescn.pt
tel:	+ 351 2 2094300
fax:	+ 351 2 2008487	
