Sahota's Dynamo system [11] pits two remote control cars against one another in a game of 1 vs. 1 soccer. All of the behaviors are hard-wired, as is the decision mechanism used to choose among the behaviors. Although this approach worked well for a specific task, ML is needed in order to avoid the cumbersome task of hard-wiring the robots for every new situation.
Also working within the Dynamo project, Ford et. al. [4] began learning the decision mechanism for these same hard-wired behaviors. They used a Reinforcement Learning (RL) approach, with sensory predicates used to reduce the size of the state space in a natural way. Our predicates used as inputs to NNs are somewhat analogous to their use of predicates for RL. So far all of the work in the Dynamo project has used Sahota's hard-wired behaviors.
On the other hand, Asada has used RL to learn a shooting behavior [1]. His framework differs from Dynamo in that it uses on-board sensors as opposed to an overhead camera. Again by reducing the state space significantly, he was able to use RL to learn to shoot a stationary ball into a goal. His best result in simulation is a 70% scoring rate. He has also done some work on combining different learned behaviors with a separate learned decision mechanism on top [2].
Our work builds on previous work by learning a more difficult behavior: shooting a moving ball into a goal. Our long-term goal is to create a team of agents with entirely learned behaviors and decision mechanisms that will be able to compete in the planned RoboCup event at IJCAI '97 [5]. This event will have both a real-world and simulated event in which researchers from around the world will be able to compare systems and methods.