Newsgroups: rec.games.corewar,comp.ai.alife
From: Robert@buchanan.demon.co.uk (Robert Macrae)
Path: cantaloupe.srv.cs.cmu.edu!das-news2.harvard.edu!news2.near.net!howland.reston.ans.net!news.sprintlink.net!peernews.demon.co.uk!buchanan.demon.co.uk!Robert
Subject: Re: Evolving CoreWar Warriors (was Pointers in pMARS)
References: <mreddy-0103951200330001@mreddy.comp.glam.ac.uk> <3j81v3$7ng@senator-bedfellow.MIT.EDU> <mreddy-0903950716040001@mreddy.comp.glam.ac.uk> <3ki5qe$7ka@falcon.ns.net> <mreddy-2003952048370001@mreddy.comp.glam.ac.uk>
Organization: Buchanan Partners Limited
Reply-To: Robert@buchanan.demon.co.uk
X-Newsreader: Demon Internet Simple News v1.29
Lines: 99
X-Posting-Host: buchanan.demon.co.uk
Date: Thu, 30 Mar 1995 11:20:36 +0000
Message-ID: <796562436snz@buchanan.demon.co.uk>
Sender: usenet@demon.co.uk

I'm interested in Genetic Algorithms, which I've been using in developing 
financial forecasts. I think that Corewars is a most appropriate testbed 
for this kind of work because the competitive nature of the problem mirrors 
that in some very real financial environments. I'd definitely like to see 
the results! I've been thinking about writing a GA or similar to test 
combinations of properties in papers, but more of that later.

> This might not sound like much, but he started with seeding warriors which 
> were generated completely at random. Many warriors lost every battle in
> the first generation (as might be expected) but the trend was towards more
> draws and less wins and losses as each generation progressed.

The tendency towards more draws looks a bit worrying; it suggests that you 
may tend to hit a plateau in the warrior development, with warriors which 
don't kill themselves, but don't do anything very offensive either. This 
kind of plateau behaviour has been our major concern with practical use 
of these techniques.

> The successful beast is only about eight instructions long, and seems to
> copy a random memory location backwards throughout the core. Apparently,

This sounds like a bomber variant which is tuned against imps. It will beat 
a wide range of warriors, but it would lose to a "Paper" -- a self 
replicating program which runs many copies of itself in parallel and hence 
has redundancy to handle occasional bombing. This aspect of Corewars (that 
a mediocre paper will beat a good bomber but a good paper will lose to a 
mediocre scanner, etc) makes _populations_ of programs more interesting 
than an individuals since they can encompass several different families of 
programs. 

[ One project I'm going to attempt, real soon now, is to code up the game-
  theory code to calculate the equilibrium population given the score matrix 
  for a set of warriors. This would tell you, for each warrior, the 
  percentage of the time it would be used under the best mixed strategy for 
  the game. In an evolutionary context, it tells you (I believe) the 
  percentage it would make of a stable population. This measure shows which 
  warriors can beat other "Good" programs, rather than ones which are merely 
  efficient lame-duck killers, which would make it a good adjunct to any GA 
  work.]

Now (getting back to the point, I hope) I think the dominance of the bombing 
warrior may chiefly reflect a population problem -- the difficulty of 
evolving replicators. A replicator must both copy itself to a new location 
AND start a new thread there; a near miss won't do. This seats replicators 
in very steep local minima which do not have basins of attraction around 
them, so are very hard for any stochastic minimisation program to find. 
If there are no replicators in the population, then you will only evolve 
variants of bombers (imp rings being even harder to evolve than 
replicators).

To evolve really competitive warriors you'd need to ensure that there are 
always at least one representative of each type of warrior present in the 
population. One way to do this is to include papers and imp rings in the 
starting mix, but I suspect you'd see very few offspring which were better 
than their parents.

Another tack is to change the level at which the genes are expressed in 
Redcode. The most obvious choice (which I assume you've taken) is to map 
genes directly onto Redcode so that any combination of codes, modes and 
constants is available. Our  experience has been, however, that there is 
considerable value in _adding_ higher level constructs to the list of 
available elements so that useful combinations appear more often. In a 
redcode context this would mean adding genes which translate into groups 
of opcodes. For example, the key code from a Silk-type self-replicating 
code is:

	SPL     target, 0
	MOV     >-1,    }-1             (Or similar).

This accomplishes the self-replication, IF located at the start of the 
warrior and IF executed by as many processes as the code length. Now 
clearly this combination is very hard to find through pure random search, 
but if the above code fragment is available as a single gene (with one 
parameter, target), replicators become much easier to generate. I was 
considering using stochastic minimisation on a very restricted subset 
of warriors which consist soley of such replication opcode-pairs 
interspersed with MOV x, {y bombing attacks, but so far hand coding has
been easier.

Some people look at this type of seeding as cheating. Clearly if you seed 
with constructs like this, the creation warriors  which merely survive 
becomes much less impressive. I'd advocate it, however, for three reasons. 

First, you are not making any part of the search space unobtainable, as 
all the basic opcodes are still available. The GA can still produce any 
legal program, you are only concentrating the search in a-priori plausible 
areas. 

Second, the GA process depends on abstraction of structure which can be 
genetically manipulated. It is up to us how we do this, and I'd back 
skilled abstraction against unskilled any day. If skilled examination of 
existing warriors shows that there are recognisable building-blocks which 
involve several co-ordinated opcodes, then make these building-blocks 
available. 

Third, though the success in evolving viable warriors from the redcode 
level up is pretty impressive, the real test in Corewars or finance, 
is how far down the road from viable to ideal you can tread...
 
