==============================================================================

12:00, Thursday 15 May 1997, Wean 1302

		      Evaluating Robot Learners
		 Jeremy Wyatt, Birmingham University

It is difficult to define a coherent experimental method for robot
learning. This is partly because an observed phenomenon may be caused
in a number of ways. The robot's behaviour may be the product of the
learning algorithm; the robot's initial knowledge; some property of
the robot's sensors; the environment; or of an interaction between
some subset of these. How do we design robot learning experiments so
as to generate meaningful results in the face of such complexity?
Sensibly enough the answer hinges on the evaluation methods used. I
shall argue that to disambiguate the source of an error or behaviour
we need to employ several forms of evaluation. In particular I draw
distinctions between external and internal measures of a robot's
performance; and also between quantitative and qualitative forms of
description. I will present the techniques applied to some experiments
with robot called Asterix which learns to push boxes on the basis of
punishments and rewards.