
Genetic Algorithms Digest   Tuesday, December 24 1991   Volume 5 : Issue 40

 - Send submissions to GA-List@AIC.NRL.NAVY.MIL
 - Send administrative requests to GA-List-Request@AIC.NRL.NAVY.MIL

Today's Topics:
	- Re: GAs and Very Fast Simulated Re-annealing
	- Machine Learning Workshop 1992: ``Biases in Inductive Learning"
	- GAs and dynamic environment

**********************************************************************

CALENDAR OF GA-RELATED ACTIVITIES: (with GA-List issue reference)

 Canadian AI Conference, Vancouver, (CFP 1/7)                 May 11-15, 1992
 COGANN, Combinations of GAs and NNs, @ IJCNN-92 (v5n31)      Jun 6,     1992
 10th National Conference on AI, San Jose, (CFP 1/15)         Jul 12-17, 1992
 FOGA-92, Foundations of Genetic Algorithms, Colorado (v5n32) Jul 26-29, 1992
 COG SCI 92, Cognitive Science Conference, Indiana, (v5n39)   Jul 29-1,  1992
 ECAI 92, 10th European Conference on AI (v5n13)              Aug  3-7,  1992
 Parallel Problem Solving from Nature, Brussels, (v5n29)      Sep 28-30, 1992

 (Send announcements of other activities to GA-List@aic.nrl.navy.mil)

**********************************************************************
----------------------------------------------------------------------

Date: Fri, 20 Dec 91 13:50:40 EST
From: 449Smi <brosen@cis.udel.edu>
Subject: Re: GAs and Very Fast Simulated Re-annealing

   This message replies to both Davis and Eshelman's  comments,
   (GALIST  v. 5 #37) on our paper "GAs and very fast simulated
   re-annealing: a comparison", (GALIST  v.  5  #36),  and  ad-
   dresses some of the issues that they have brought up.

   Our first comments is a reply  to  David  Davis's  comments.
   David  Davis  brings up a good point that function F1-F7 (we
   mention only F1-F5) should not be used to assess GA  perfor-
   mance  because  F1-F7  are  inadvertently  biased  to  favor
   mutation-based systems, as the optimal  solutions  are  cen-
   tered  in  the search space.  Although we have not performed
   tests to verify this conjecture, we  agree  that,  as  Davis
   suggest  in his comment #2; shifted versions of problems F1-
   F7 should be used.  New functions that shift the optima  off
   the  center  of  the  range should be incorporated in future
   tests.  However, we would like to point out that VFSR is not
   a  mutation-based  system,  but one based on importance sam-
   pling. As no encoding/decoding mechanism is  used  in  VFSR,
   this shifting should not affect or degrade its performance.

   Our next comments reply  to  Larry  Eshelman  and  J.  David
   Schaffer,  in which they present empirical and ergodic argu-
   ments on our claims.

   In the empirical argument they argue that  although  we  did
   not  list all parameters used in our function optimizations,
   and we chose GA parameters that were inefficient  for  these
   functions. In our simulations we used the default, heuristi-
   cally determined parameters of the GA UCSD simulator  (based
   on  Genesis 4.5), which is publicly available by ftp, and is
   widely known and used. Although we  only  explicitly  listed
   populations sizes, there were many options/parameters values
   to adjust, including population size, crossover rate,  muta-
   tion  rate  and genome length to name a few. No GA parameter
   fine tuning was done by us, as the simulator set the default
   parameters  of  the  function  f0-f5  with sets of different
   parameters. Thus, the UCSD GA simulator had  performed  some
   prior tuning of the GA algorithm for f0-f5.

   Although we do not comment on the inefficiency of large  po-
   pulation  sizes, we do note that one of the major advantages
   of GAs are that they are inherently parallel, and that there
   is  little  or  no  overhead simulating large populations on
   parallel computers (such as the  CM)  available  today.  (We
   also gave an outline describing that VFSR too is paralleliz-
   able.)

   In addition to speed, accuracy of results is important  part
   of  an  optimizer.   Larry  Eshelman  and  J. David Schaffer
   present results of three specific GA  implementations.  They
   cite  specific parameter values, and use gray coded function
   parameters, which they claim produces  better  results  than
   binary  coding.   We  comment that when using binary GA bits
   string representations, unless  the  strings  are  of  equal
   length  as  those  used  to hold real value parameters (e.g.
   float = 32 bits, doubles = 64 bits), equivalent  representa-
   tional  levels  of  accuracy  cannot  be  achieved, but only
   coarser approximations of the optima can be found. Optima do
   not  always  lie  precisely  at 0, (this is discussed in our
   first comments above). (Gray scale representations  may  re-
   quire  longer  bit strings to achieve equal representational
   levels of accuracy.) As stated in our paper, bit lengths  of
   f1-f5  ranged  from 8 to 17 bits were used in our GA simula-
   tions (f0 used 32 bits). One would  expect  to  have  slower
   convergence  times  when using longer bit strings.  However,
   it is clear that Eshelman and Schaffer have shown from their
   examples  that  GA performances can be improved dramatically
   using specific versions of GAs.

   Also for most problems, there  is  a  natural  (or  imposed)
   scale of accuracy required, which builds in a natural grain,
   which is often taken advantage of in VFSR, by shunting  over
   to  a  local  code,  usually  the Broyden-Fletcher-Goldfarb-
   Shanno (BFGS) algorithm, to finish off the last few  decimal
   places  (e.g. 10e-12 to 10e-20).  However, this was not done
   in our paper.  We believe that VFSR is  the  proper  way  to
   handle  "common-sense" use of resources, e.g., starting with
   some statistical ergodicity, and then turning over  to  more
   optimal  fitting  procedures, rather than trying to build in
   short-circuits from the beginning.  However, once a particu-
   lar  system is somehow bounded, e.g., algebraically, then we
   would agree that it would make more sense to  use  an  effi-
   cient code known to work well in such a system.

   With regard to the ergodic argument we do not  believe  that
   the  desirable  property  of  ergodicity has to be defended.
   Suffice it to say, that when dealing with an unknown  system
   (to  which  we  addressed  our paper), it is nice, sometimes
   necessary, to be sure that the best fit is  being  obtained.
   The issue of the efficiency of ergodic techniques is another
   matter,  and  here  in  fact  we  agree  with  Eshelman  and
   Schaffer,  i.e.,  that  this  very  likely can be algorithm-
   dependent and system-dependent.  However, this  already  as-
   sumes that properties of the system are known.

   The thrust of SA, from the original 1953  use  in  path  in-
   tegrals to the 1983 paper in application to circuits, in ad-
   dition to stressing ergodicity, also stresses the concept of
   importance  sampling,  the  use  of the cost function to in-
   crease the efficiency of the fit, yielding exponential gains
   over  purely  (uniform)  random  searches.  It seems that at
   least some GA critiques argue against ergodicity with  exam-
   ples  that  really  apply to such uniformly random searches,
   but do not particularly account for the importance  sampling
   capability.

   Larry Eshelman and J. David Schaffer argue that it is diffi-
   cult to compare GA and VFSR since both algorithms are sensi-
   tive to parameters and internal variations.  Our  experience
   has  been that tuning can often be very important to getting
   the most efficient fits, especially in nonlinear  stochastic
   systems.  Much time and effort may be needed to find the ap-
   propriate parameters, often building on others experience to
   find  them.   For  example,  this  has  been done in several
   specific complex systems using VFSR, as  referenced  in  our
   paper, to find be to find the most precise fits in a minimum
   of time.

   However, while we agree that GA are sensitive to many param-
   eters,  VFSR  is  comparatively insensitive, with relatively
   few values to tune.  Although some prior  knowledge  of  the
   system  can be used to enhance the efficiency of the search,
   it is not necessary to tune to (statistically) guarantee  an
   optimal fit.  In fact the major parameters typically used in
   our paper are the finite ranges on the inputs using  a  spe-
   cially constructed generating function for this purpose (one
   of the differences between VFSR and other simulated  anneal-
   ing  algorithms),  and these were external constraints.  For
   many real problems, this is fine and in fact useful, instead
   of  wasting resources out in infinity-land.  This makes VFSR
   a very robust system, which has been tested and used on wide
   variety  of  functions,  although  it  obviously  cannot  be
   claimed to be the best optimizer for  every  test  function.
   For  example,  we  were shown a simplex routine that did in-
   credibly well on f0, up to  about  2-3  significant  figures
   (because  the  large-resolution parabolic well allowed it to
   find a few more valleys).

   As is pointed out by many proponents of  many  fitting  rou-
   tines, once "confidence" is attained for a given approach to
   fitting a given system, it only makes sense to use the  most
   efficient  process.   However  we  agree  with  Eshelman and
   Schaffer when they point out that more work is needed on the
   study  of  the  domains  that GAs (and other algorithms) are
   useful for.

   Each approach has its utility, e.g., even the Newton method,
   over  others.   Therefore,  in a specific system it is worth
   examining at least several alternatives before  doing  "pro-
   duction"  runs.  However, if one has a new problem to solve,
   we believe that a SA method, even if  slow,  can  offer  the
   best guidance to the location of minima that might be missed
   by the other techniques.

   Lastly, we would like to  thank  Larry  Eshelman,  J.  David
   Schaffer  and  David  Davis for their comments and review on
   our paper, (although this is not  meant  to  imply  that  we
   agree with all their conclusions).

   Bruce Rosen and Lester Ingber

------------------------------

From: spears@AIC.NRL.Navy.Mil
Date: Fri, 20 Dec 91 13:43:11 EST
Subject: Machine Learning Workshop 1992: ``Biases in Inductive Learning"

                          CALL FOR PAPERS
        Informal Workshop on ``Biases in Inductive Learning"
        To be held after the 1992 Machine Learning Conference
            Saturday, July 4, 1992   Aberdeen, Scotland

	All aspects of an inductive learning system can bias the learn-
   ing  process.   Researchers  to  date have studied various biases in
   inductive learning such as algorithms,  representations,  background
   knowledge,  and  instance orders.  The focus of this workshop is not
   to examine these biases in isolation.  Instead, this  workshop  will
   examine how these biases influence each other and how they influence
   learning performance.  For example,  how  can  active  selection  of
   instances  in concept learning influence PAC convergence?  How might
   a domain theory affect an inductive learning  algorithm?   How  does
   the choice of representational bias in a learner influence its algo-
   rithmic bias and vice versa?

	The purpose of  this  workshop  is  to  draw  researchers  from
   diverse  areas to discuss the issue of biases in inductive learning.
   The workshop topic is a unifying theme for  researchers  working  in
   the  areas of reformulation, constructive induction, inverse resolu-
   tion,  PAC  learning,  EBL-SBL  learning,  and  other  areas.   This
   workshop  does  not  encourage papers describing system comparisons.
   Instead, the workshop encourages papers on the following topics:

    -  Empirical and analytical studies comparing different  biases  in
       inductive learning and their quantitative and qualitative influ-
       ence on each other or on learning performance

    -  Studies of methods for  dynamically  adjusting  biases,  with  a
       focus  on the impact of these adjustments on other biases and on
       learning performance

    -  Analyses of why certain biases are more suitable for  particular
       applications of inductive learning

    -  Issues that arise when integrating new biases into  an  existing
       inductive learning system

    -  Theory of inductive bias

   Please send 4 hard copies of a  paper  (10-15  double-spaced  pages,
   12 point font) or (if you wish to attend, but not present a paper) a
   description of your current research to the workshop chair:

	       Diana Gordon
	       Naval Research Laboratory, Code 5510
	       4555 Overlook Ave.  S.W.
	       Washington, D.C.  20375-5000   USA

   Email submissions to gordon@aic.nrl.navy.mil  are  also  acceptable,
   but  they  must  be  in  PostScript.   FAX  submissions  will not be
   accepted.  If you have any questions about the workshop, please send
   email  to  Diana  Gordon at gordon@aic.nrl.navy.mil or call 202-767-
   2686.

   Important Dates:

      March 12 - Papers and research descriptions due
      May 1 - Acceptance notification
      June 1 - Final version of papers due

   Program Committee:

      Paul Vitanyi, CWI and University of Amsterdam
      Devika Subramanian, Cornell University
      William Spears, Naval Research Laboratory
      Jude Shavlik, University of Wisconsin
      Larry Rendell, University of Illinois
      Dennis Kibler, University of California at Irvine
      Diana Gordon, Naval Research Laboratory

------------------------------

From: Claude Muller <cmuller@comms.eee.strathclyde.ac.uk>
Date: Thu, 19 Dec 91 20:11:49 GMT
Subject: GAs and dynamic environment

   Hello all,

   Has anybody look at GAs for resource allocation problems in a dynamic
   environment ? In order to solve the 'dynamic environment' problem, we 
   intend to investigate the following directions:

    1. Specific representations: Rather than a change in the fitness
   function, Is it possible to act directly on the representation whenever
   the problem to solve is altered ?

       Modification of the crossover: we will use a repair algorithm (as a
   local search in fact). Can this solve one part of the "dynamism" problem ?

    2. We view the resource allocation problem as a constraint satisfaction
   problem (CSP):

	Has anyone been across (or produced) some work in this area: CSP-GAs?

	What are 1) the best representations for this type of problem 
		 2) the best evaluation techniques?

	Any comparisons with different techniques (ie: neural nets, heuristic,
	simulated annealing,...) for large CSP problems?

	 As said previously, we will investigate 'repair-crossover' in order
	 to solve the constraint aspect of the problem, has anyone done some
	 work on the subject ? Is is relevant ?

     3. One of the objectives is to produce a fast answer:

	 Has anyone applied GAs to near real time CSP resource allocation 
	 problems ?

	 Can an exponential evaluation technique help to achieve this aim ?

    Thanks for your comments.
    Claude Muller

------------------------------
End of Genetic Algorithms Digest
******************************
