HELP COMPETITION          Jocelyn Paine and Edmund Chattoe, October 1992


The Society is using the Eden microworld for testing out approaches to
learning and planning. Despite its simplicity, Eden provides a
challenging environment for AI programs. We would like to encourage our
members both to take up this challenge, and to meet, if only by Email,
AI-ers from other universities. To this end, we asked the Poplog Users'
Group to adopt Eden as their next Poplog competition: they run these at
roughly yearly intervals. They have agreed, and the competition has been
advertised to all members of the Users' Group. This HELP file contains a
copy for Oxford AI Society members.

This file describes the rules for the competition. For details of Eden
itself, see TEACH EDEN.


The competition
---------------

The competition consists of three stages. In the first stage, you run
your bug on the ``training worlds''. These are a set of twenty extremely
simple microworlds, each containing a single object or pair of linked
objects (like doors and keys). You can run your Bug through them as many
times as you like, with the objective of teaching it the properties of
these basic objects. There are also some more advanced training worlds,
called problem worlds, which are similar but not identical to those to
be used in the competition. They contain the same types of object in the
same kinds of combinations, but differ somewhat in their exact
geography. Again, there are twenty of these.

You can access the training worlds by using the names 'worlds:tw1'
through 'worlds:tw20'; thus:
    run( 'mybug', 'worlds:tw9' );
will run the bug defined in file mybug.p on training world 9. Similarly,
you can access the problem worlds by using the names 'worlds:pw1'
through 'worlds:pw20'.

To enter the competition, you must send us the source code of your
``think'' and ``bugdead'' procedures. This should be in Poplog Pop-11,
Prolog, or Lisp: we cannot guarantee to provide compilers for C and
other external languages. In the second stage, the beginning of the
competition proper, we will put this source onto our machine and run it
on a series of competition problem worlds. For example, you may have to
open a chest to get the key to a door that shields the food. Oxford AI
Society members can enter their bugs in the competition by Emailing
Jocelyn Paine at POPX@OX.VAX.

The idea then is that we will take your Bug, trained on the training
worlds, and put it into the first competition world. When it dies,
either through eating the food, running out of energy, or being killed
by a fatal object, it can elect either to stay in the same world and
have another go, or to pass on to the next one in the series. It
indicates its decision by the result passed back from ``bugdead''.

The reason for allowing Bug to return to the same world is to allow it
some more training time. However, such time carries a penalty. Bug's
ranking in the competition is determined by its ``success score'' (the
number of worlds in which it succeed in finding and eating the food),
normalised by the total number of cycles spent. Thus, if you fail to
find the food and, sadder but wiser, decide to try again, you will be
penalised for the extra number of cycles spent in gaining that wisdom.
So stage two definitely rewards well-trained bugs. The final section of
this file described the scoring in more detail.

Since learning is one of the most difficult problems in AI, we will
accept bugs whose behaviour has been entirely pre-programmed, perhaps in
a rule-based system. The competition pack describes the properties of
all the objects encountered during stage two. Together with the training
worlds, these will give you all the information you need to write such a
Bug. Since this kind of Bug would gain no benefit from being
reincarnated in a competition world that had just killed it, it will
probably elect to proceed to the next world after each death. We hope
this arrangement will allow both novices and experienced AI-ers to
take part.

Stage three of the competition is harder, and puts the emphasis on
learning. This time, we will run your Bug through a sequence of worlds
where ``new'' objects will exist which Bug will not have encountered
before. Thus you might be presented with a portable quicksand --- like
an ordinary quicksand, it will drown you, but you can also pick it up
(like a key).

This final stage hopes to separate genuinely ``intelligent'' Bugs
from the others. It is quite possible however that none of the Bugs
will succeed.


The rules
---------

Because this competition is designed to encourage the development of AI
techniques, it is very difficult to ``cheat'' and there are few rules.
Those ``rules'' which do exist are typically constraints on the way that
you must write 'bugdead' and 'think' to ensure a proper interface. The
only other rule is the way that the score is calculated.

1) The judges' decision is final. They reserve the right not to consider
programs written in languages other than those specified above.
Reasonable attempts will be made to fix compatibility problems - as
opposed to programming errors - and competitors will be informed of
these problems so they can cure them if possible. For this reason, we
advise you to avoid non-standard programming that may be dependent on
the properties of a local machine. The purpose of this rule is not to
make life difficult, but to ensure that we are fair to all competitors by
not devoting undue attention to programs that don't work properly! If
you are unsure about whether something will run on our system, ask in
advance and we will try to help.

2) The procedure 'think' must take as input ``sense data'' in the form of
... and output a single action in the form ...

3) The procedure 'bugdead' must take as input ... and output ...

4) All competitors will be provided with the same set of twenty training
worlds and twenty examples of problem worlds. These example problem
worlds will contain the same objects and types of problems as those
forming stage two of the competition, though the objects may have a
different spatial arrangement. Competitors may generate and use any
other worlds they choose for training and testing purposes.

5) Stage two of the competition will be run on twenty problem worlds
containing the same objects as the training worlds. These objects are as
follows:
        + = food        - eat this to gain energy.
        * = boulder     - impossible to move through.
        @ = rock        - can be smashed with hammer (T).
        # = door        - can be opened with key (k).
        Q = quicksand   - drowns you unless you immediately move off it.
        T = hammer      - smashes a rock.
        k = key         - opens a door.
        : = narrows     - only passable if you are not carrying
                          anything.

Their properties are fully described in TEACH EDEN.

6) Stage three of the competition will be run on ten worlds containing
``new'' objects.

7) The scoring of the competition will be done on the basis of a single
run though this set of worlds. The point score for each world i equals
the total number of energy points remaining for all lives in world i
divided by the number of lives in world i. The point score for the whole
competition is simply the sum of point scores for each world.

Example
-------

Suppose Bug1, a good one, manages three worlds in 12, 13 and 20 turns,
it scores:

(488/1)+(487/1)+(480/1)=1455

Bug2 on the other hand dies on the first world, elects to go forward,
has two goes at the second world and dies both times. Finally it elects
to go onto the third world, dies on the first attempt and succeeds on
the second in 75 turns. It scores:

(0/1)+(0/2)+(425/2))=212.5

Clearly, this scoring puts a premium on finishing worlds rather than
finding optimal solutions. In practice, since the worlds get more
complex, it is unnlikey that bugs could fail on early worlds and succeed
on later ones but the rules do not forbid them to try!
