
Genetic Algorithms Digest    Friday, 27 July 1990    Volume 4 : Issue 13

 - Send submissions to GA-List@AIC.NRL.NAVY.MIL
 - Send administrative requests to GA-List-Request@AIC.NRL.NAVY.MIL

Today's Topics:
	- Message from moderator
	- Deceptive Problems and Reflective GAs
	- THEOREM: fully deceptive problems occur only across hamming cliffs
	- RE: Using GA's for concept learning
	- Warning about random number generators in genetic algorithms

******************************************************************************

CALENDAR OF GA-RELATED ACTIVITIES: (with GA-List issue reference)

Conference on Simulation of Adaptive Behavior, Paris (v3n21)  Sep 24-28, 1990
Workshop Parallel Prob Solving from Nature, W Germany (v4n5)  Oct 1-3,   1990
2nd Intl Conf on Tools for AI, Washington, DC (v4n6)          Nov 6-9,   1990
4th Intl. Conference on Genetic Algorithms (v4n9)             Jul 14-17, 1991

(Send announcements of other activities to GA-List@aic.nrl.navy.mil)

******************************************************************************

Date: Fri, 27 Jul 90
From: Alan C. Schultz (GA-List Moderator)
Subject: various things...

    Well, it looks like the recent workshop has finally stirred us into
    some stimulating discussions!  This issue starts with two submissions
    on "deceptive problems".

    Reminder: If you participated at the workshop, please send a brief
    abstract of your presentation.  I will put these together and send
    out several special issues of GA-List with the abstracts.

    --Alan

----------------------------------

Date: Thu, 26 Jul 90 17:35:36 EDT
From: gref@AIC.NRL.Navy.Mil ("John J. Grefenstette")
Subject: Deceptive Problems and Reflective GAs

  At the recent Foundations of GAs Workshop there was much discussion of
  deceptive problems.  Some interesting approaches to these problems were
  discussed, from searching for ways to transform the representation, to
  using other forms of GAs to solve them.  One problem with the latter
  approach is that the new GAs may themselves be deceived by other
  problems. I have a suggestion that seems to avoid this difficulty.  You
  might call it a "Reflective GA".

  Consider the following algorithm: maintain two separate populations of
  size N, call them A and B.  At each generation, evaluate all structures
  in both populations, apply an ordinary GA to population A, then form a
  new population B by taking the binary complement of each element in A.
  Keep track of the best structure to appear in either population.

  This algorithm solves any problem solved by an ordinary GA with
  population of size N, in at most twice the amount of time.  It also
  solves any of the proposed fully deceptive problems, since they have the
  property that the true optimum is the binary complement of a local
  optimum to which the ordinary GA is led.

  There are many possible variations on this theme.  For example, it may
  be desirable to allow recombination across populations A and B for
  "partially deceptive" problems.

----------------------------------

Date: Fri, 20 Jul 90 10:10:45 MDT
From: whitley@ravel.CS.ColoState.EDU (darrell  whitley)
Subject:

    It seems to me the following proof by contradiction
    holds water.  Anyone want to comment on this.  

    Darrell Whitley  whitley@ravel.cs.colostate.edu

    THEOREM: A fully deceptive problem of any specific "order" can occur
    only across hamming cliffs.

    Proof by Contradiction:

    Assume full deception exists across hyperplanes that share at least
    one bit in common.  Choose one bit that is shared by the true
    optimum and the deceptive optimum.  Let 1-T represent the bit
    that agrees with the true optimum and let 1-D represent the bit
    that agrees with the deceptive optimum.   These 
    bits have the same value (be it 0 or 1). The problem is not
    fully deceptive at a specific "order" unless the hyperplane
    *...*1-T*...* < *...*1-D*...*, but this is impossible, since they
    are the same bit and have the same value.  

    (If you do not agree that full deception implies
    *...*1-T*...* < *...*1-D*...* then consider the following
    additional argument.)

    Let C represent the complement
    of bits 1-T and 1-D (which are in fact the same bit). 
    To be deceptive, it is necessary that  *...*1-T*...* < *...*C*...*,
    but this also implies that *...*1-D*...* < *...*C*...*.  Therefore,
    if deception exists, it requires that the deceptive solution
    have the bit C and not 1-D.  Therefore the proposed
    deceptive problem containing the bit 1-D is not fully deceptive.

    QED 

    This argument can be used recursively to show that any problem
    proposed to be deceptive must be "reduced" to a lower order
    deceptive problem until the point that the reduced proposed problem
    resides at a hamming cliff.   I think this restriction makes the concept
    of deceptiveness a bit more "brittle."    As Dave Schaffer has
    pointed out, the "brittleness" of deception may turn out to
    be rather important.  However, if the deception exists over arbitrary
    bit locations in a string, finding the deceptive bits could involve
    searching a space of 2**L, since it involves finding a bit mask
    that indicates the location of the deception.   On the other hand,
    remapping the bits to ANY new mapping that does not place the
    critical bits at a hamming cliff will reduce the deception in those
    specific locations (but will perhaps create deception elsewhere.)

-----------------------------

Date: Fri, 27 Jul 90 13:11:29 EDT
From: spears@AIC.NRL.Navy.Mil
Subject: RE: Using GA's for concept learning

	Back in (GA-List v4n8) I summarized some of the
	work that I'm doing with GAs at the NRL AI Center.
	At that time I mentioned that I hoped that my comments
	would spark debates, questions, and interest. It
	appears that I was at least partially successful -
	there has been much discussion concerning the use
	of GAs for concept learning.

	I've decided to add a few responses to the discussion
	that has resulted...

  >>>  John Havener wrote: (GA-List v4n9)
  >>>
  >>>      I need to apply GA to learn from a vast database of historical
  >>> readings.
  >>> ........ [rest deleted]
  >>
  >> Michael Hall replied: (GA-List v4n10)
  >>
  >>     I advise against using a GA (or neural net for that matter) to learn
  >> such concepts. There are a family of algorithms that are tuned to solve
  >> this problem, and you will be hard pressed to beat them with a reasonable
  >> amount of CPU.
  >>          ..........[rest deleted]
  >
  > Riyaz Sikora replied: (GA-List v4n11)
  >
  > In some cases a GA can be more effective... [rest deleted]


	I would first like to address some comments made by Michael
	Hall. First, although it is true that many of the current
	concept learning programs work very well on many problems,
	there are still concepts they do not learn well. These
	concept learning programs generally incorporate strong
	biases in order to more effectively search the hypothesis
	space. When the biases do not match the target concept well
	the concept learning program performs poorly.

	One of the major problems with many of the concept
	learners is their fixed bias. A recent dissertation
	by Diana Gordon of UMD has illustrated the advantages
	of flexible bias in concept formation. However, even
	with this flexibility, it is important to be able to
	globally search the hypothesis space efficiently.
	As mentioned by Riyaz Sikora, one of the advantages of
	using GAs is that they perform more global search than
	more conventional search techniques. 

	Previous attempts have attempted to use GAs for concept
	learning, with rather poor results. In fact, 
	Mr. Hall cites Quinlan's 1988 comparison article
	in which GA concept learner lost to C4.5.
	C4.5 was run against Boole, which was a classifier
	system. C4.5 is the latest in a long line of ID-like
	systems. Boole was a first cut. Subsequent studies
	(Lashon Booker and S. Sen) have improved on Boole's
	results. Classifier systems are tricky beasts, and
	I'm not ready to conclude that they are not good
	for concept learning.

	Since I'm not a classifier system expert, my
	approach to doing concept learning with GAs was
	via the Pittsburgh approach, in which each individual
	in the population represents a complete hypothesis.	
	Our recent paper on using GAs for concept learning includes some
	comparisons with ID5R and C4.5. Although both ID5R and
	C4.5 often perform well there are some simple target cases
	where the decision tree language bias, or the information
	theoretic algorithmic bias, are not good biases. In these
	cases, the GA can win. From an effectiveness viewpoint,
	then, GAs CAN do concept learning.

	As mentioned by others, though, there is always the
	performance aspect (ie., the GAs take a long time).
	My system has certainly not solved this problem.
	However, I would like to point out that there is
	nothing inherently unreasonable in the complexity
	of a GA, and in fact a GA is extremely parallelizable.
	However, even if GAs NEVER perform as quickly as
	other systems, they may be more effective.

	In summary, I would like to mention that I'm
	certainly not minimizing the importance of the
	more conventional concept learning paradigms. These
	paradigms work well on many target concepts, they
	are robust, and they are easy to use. Mr. Hall is
	quite correct in praising the qualities of these
	highly tuned algorithms.

	I disagree only in the conclusion that classifier
	systems (and GAs) are not good for concept learning.
	Certainly, our work indicates otherwise, and Riyaz
	Sikora has demonstrated feasibility as well. GAs
	rarely perform well on simple problems, in comparison
	with traditional techniques. However, in complex
	multimodal spaces the global search characteristics
	of GAs often win.

--------------------------------------

Date: July 23, 1990
From: Raymond Greenwell (matrng@hofstra.bitnet)
Subject: Warning about random number generators in genetic algorithms

  I recently encountered a problem using the random number generator on p.
  334-335 of David Goldberg's book "Genetic Algorithms in Search, Optimization,
  and Machine Learning."  The generator is from p. 171-173 of Donald Knuth's
  "The Art of Computer Programming, Vol. 2: Seminumerical Algorithms," 2nd
  edition.  Knuth programs the generator in FORTRAN using integers.  He adds,
  "It would be possible to work directly with floating point numbers instead
  of integers by making appropriate changes to these programs, provided that
  the computer's floating point arithmetic is sufficiently accurate to give
  exact results in all the computations required here."  The version in
  Goldberg's book makes these changes and works fine in TurboPascal on the
  IBM PC.  Unfortunately, it does not always perform satisfactorily in VAX
  Pascal.  When seeded with the number 0.5, all that is ever output is a
  sequence containing 0's and 0.5's.

  One solution to this problem on the VAX is to change oldrand from an array
  of real to an array of double, the double precision real type in VAX Pascal.
  Also, the variable new_random in procedure advance_random and the variables
  new_random and prev_random in the procedure warmup_random must be changed
  to double.  Finally, in the function random, the line random:=oldrand[jrand]
  should be changed to random:=sngl(oldrand[jrand]), where sngl is the VAX
  Pascal function which changes double precision reals to regular reals.

  Users of random number generators may also be interested to know that
  a recent survey of random number generators (Stuart L. Anderson, "Random
  number generators on vector supercomputers and other advanced architectures,"
  SIAM Review (32:2), June 1990, p. 221-251) found that, for many purposes,
  the simple generator x(n+1)=x(n)*16807 mod (2^31-1) performed exceptionally
  well on nonvector computers.  Here is a version written in VAX Pascal:
    const two31m1 = 2147483647; {2^31-1}
    var iseed: integer; {This needs a starting value of an integer between
			 0 and 2^31-1}
    function randum: real;
      var temp: double;
      begin
	temp:=16807*iseed;
	while temp>two31m1 do temp:=temp-two31m1;
	iseed:=round(temp);
	randum:=iseed/two31m1;
      end;


--------------------------------
End of Genetic Algorithms Digest
********************************

