Newsgroups: comp.ai.neural-nets
Path: cantaloupe.srv.cs.cmu.edu!das-news2.harvard.edu!news2.near.net!news.mathworks.com!europa.eng.gtefsd.com!howland.reston.ans.net!pipex!uunet!EU.net!sun4nl!news.nic.surfnet.nl!tudelft.nl!galaxy.twi.tudelft.nl!parmul
From: parmul@kgs.twi.tudelft.nl (PM project_Bernard)
Subject: Re: NN/GA Hybrids
Message-ID: <parmul.782909574@galaxy.twi.tudelft.nl>
Lines:       51
Sender: news@news.tudelft.nl (UseNet News System)
Nntp-Posting-Host: galaxy.twi.tudelft.nl
Organization: Delft University of Technology
References: <387eht$7qa@newsbf01.news.aol.com> <parmul.782727665@galaxy.twi.tudelft.nl> <dixon.5.000A66E5@sam.neosoft.com>
Date: Sun, 23 Oct 1994 10:52:54 GMT

>>We used a GA to develop the weights and the parameters for the activation 
>>functions, such as the steepness for a sigmoid, and the radius for a 
>>RBFN, simultaneously. When doing this however, one has to avoid epistasis, 
>>for example by taking a suitable representation of the NN.

>What is epistasis ? And can you elaborate on how the representation of
>the NN is shown to the GA such that it avoids it ?

Epistatis is the term used to indicate the interdependent influence of 
stringparts of the chromosome on the fitness of that chromosome. The 
influence of a stringpart on the fitness depends on the values of stringparts
elsewhere in the chromosome. This problem is difficult to avoid. It is
usually recommended to put the stringparts with strong interdependence close
together in the chromosome (this does not solve the problem!). 
For our GA we used a representation in which the neurons (in a predetermined
network topology) in the consecutive layers where put behind each other. 

For example: take a 2-2-1 NN with sigmoid activation functions, the 
string would be: 

 N(1,1).w1 - N(1,1).w2 - N(1,1).thr - N(1,1).st - 
 N(1,2).w1 - N(1,2).w2 - N(1,2).thr - N(1,2).st -
 N(2,1).w1 - N(2,1).w2 - N(2,1).thr - N(2,1).st

N(1,1).w1 indicates the first incoming weight for the first neuron in the first
hidden layer. thr= threshold, st = steepness for the sigmoid function.
By including the activation parameters we hoped to reach satisfactory results
sooner. (This worked out well, but not as well as hoped...). 
This representation allowed us to use crossover functions that only cut 
strings between two neurons and use special mutation functions that for
example worked on all the weights in a single neuron instead of a single weight.
(The discussion about the best string representation (binary vs non-binary) is
still going on, see comp.ai.genetic)

A possible advantage (?) of the use of GAs to determine good weights 
for a NN is that one can also discontinuous activation functions 
(the gradient is not used!). One could even mix different kins of activation
functions or let the GA determine suitable activation functions. (this 
is an idea which has not been tested and therefore may be completely
ridiculous...

Hope this helps,
    Bernard Kroes

========================================================================
Bernard Kroes                   | e-mail: parmul@kgs.twi.tudelft.nl
========================================================================
-- 
========================================================================
Bernard Kroes                   | e-mail: parmul@kgs.twi.tudelft.nl
========================================================================
