Robots Move

Position Paper on Simulation for the 1998 AAAI Spring Symposium

Tucker Balch
College of Computing
Georgia Institute of Technology
Atlanta, Georgia 30332-280

Simulation isn't enough

Simulation is a powerful tool, but robotics research should be conducted on robots. In this field, demonstration on robotic hardware is required before a new idea is taken seriously. Why? Because reality is the best referee. There are dozens of ways simulation can mislead, but a robotic system running in the real world clearly establishes that its designers have overcome the myriad details of integration and physical implementation.

This is not to say that results developed in simulation are necessarily questionable, but that results on mobile robots are substantially more compelling. There is no question that simulated systems are easier to use and cheaper than robotic hardware; this is one reason for their popularity. But the availability of inexpensive robotic hardware (commercial robots are available for as little as $2000) makes it increasingly difficult to defend simulation-only studies.

But simulation is important

If implementation on hardware is crucial, why use simulators at all? The answer is that simulation can help us develop robust robotic systems more quickly than we might otherwise. Simulation speeds the development of physical systems by providing an environment for rapid prototyping. This is especially true in the development and testing of robot control systems where the simulation provides an artificial and easily configurable environment for experimentation.

As an example, our approach to multiagent robotics research at Georgia Tech follows an iterative cycle of experimentation in simulation and on mobile robots:

  1. Develop and test the robotic control system in simulation.
  2. Gather statistical performance of the system in simulation.
  3. Implement the system on mobile robots.
  4. Compare mobile robot performance with performance in simulation.
  5. Revise the simulation to reflect these differences.
  6. Go to 2.
This allows us to rapidly test and refine new behaviors before running them on mobile robots. It also enables us to evaluate group behaviors on small (1-5) teams of robots, and extrapolate performance to tens or hundreds of agents in simulation.

What price accuracy?

One might argue that we should strive to accurately simulate every detail of a robot's interaction with the environment. Research conducted in such a simulation would be just as valid as results on robots.

It's true that more accurate simulations are more useful, but they're also more expensive. Should we spend our resources on better simulations or on hardware implementation? In other disciplines the choice may not be so clear, but the point of robotics research is to build physically embodied agents. Robots move.