For the task of shooting a moving ball, the passer's behavior was predetermined: it simply accelerated as fast as it could until it hit the ball. We varied the velocity (both speed and trajectory) of the ball by simply starting the passer and the ball in different positions.
However, in a real game, the passer would rarely have the opportunity to pass a stationary ball that is placed directly in front of it. It too would have to learn to deal with a ball in motion. In particular, the passer would need to learn to pass the ball in such a way that the shooter could have a good chance of putting the ball in the goal (see Figure 4(a)).
Figure: Multi-agent learning scenarios: (a) The passer and
the shooter must both learn their tasks in such a way that they can
interact successfully; (b) The defender learns to block the
shot, while the shooter simultaneously tries to learn to score.
Our initial approach to this problem is to take the shooting template as fixed and to try to learn to pass a ball in such a way that this behavior will be successful. However, it may turn out that the passer learns to pass in a non-optimal way. In this case, it will be to the team's benefit if the shooter can learn to shoot the type of moving ball at which the passer is good at providing. Here is a situation in which an adaptive agent will have an advantage over an agent with fixed behavior.
Once the shooter adapts to the passer's behavior, the passer may in turn be able to adjust its behavior to improve their overall performance. Since each player's performance depends on that of the other, we have an interesting example of co-evolution set up here: the shooter and the passer must learn to coordinate behaviors if there is to be any success at all.