Together with researchers from the Universities of Bonn and Freiburg (Dirk Schulz, Wolfram Burgard, Maren Bennewitz, Armin B. Cremers, Dieter Fox), the Nursebot Project pursue research on tracking people in their home environment and learning models of behavioral patterns.

The People Tracking Project has devised fast algorithms for tracking variable numbers of people using laser range sensors mounted on mobile robots. At the core of our research are fast particle filters, which are statistical techniques well-syuited for the localization and tracking of moving objects. Our approach has been demonstrated to reliably estimate the right number of people and track their coordinates, even while the sensing robot moves about the environment. Below are animations that illustrate our research.

At present, the project seeks to develop methods for learning "typical" behavioral patterns of people in their home environment. Such patterns will enable us to detect deviations (e.g., domestic accidents) and to position the robot in minimally intrusive ways. We present an algorithm that learns collections of typical trajectories that characterize a person's motion patterns. Data, recorded by mobile robots equipped with laser-range finders, is clustered into different types of motion using the popular expectation maximization algorithm while simultaneously learning multiple motion patterns. Experimental results, obtained using data collected in a domestic residence and in an office building, illustrate that highly predictive models of human motion patterns can be learned. Furthermore, we propose a method for adapting the behavior of a mobile robot according to the activities of the people in its surrounding. Our approach uses the learned models of people's motion behaviors. Whenever the robot detects a person it computes a probabilistic estimate about which motion pattern the person might be engaged in. During path planning it then uses this belief to improve its navigation behavior.

Check out our animations and papers!

Animations and Videos

• Robotic tracker using particle filters for tracking multiple people. By Dirk Schulz, Wolfram Burgard, Dieter Fox, and Armin B. Cremers.

• Visualization of the paticle filtering technique for localizing and tracking people. By Dirk Schulz, Wolfram Burgard, Dieter Fox, and Armin B. Cremers.

• Video: Person moving in home environment.

• Multi-robot tracking system for a person in a home environment. Animation with full laser scans and animation with tracking scans. By Maren Bennewitz and Wolfram Burgard, University of Freiburg, and Sebastian Thrun.

• Segmented motion segments, colored according to different behavorial patterns. By Maren Bennewitz and Wolfram Burgard, University of Freiburg, and Sebastian Thrun.

• Learning motion patterns from input trajectories. By Maren Bennewitz and Wolfram Burgard, University of Freiburg, and Sebastian Thrun.

• The mobile robot Albert moves into a doorway to let a person pass by (mpg-video). By Maren Bennewitz and Wolfram Burgard, University of Freiburg, and Sebastian Thrun.
• Albert moves away from a doorway to let a person enter the corresponding room (mpg-video). By Maren Bennewitz and Wolfram Burgard, University of Freiburg, and Sebastian Thrun.

Papers

• Tracking Multiple Moving Targets with a Mobile Robot, by Dirk Schulz, Wolfram Burgard, Dieter Fox, and Armin B. Cremers, 2001 (submitted)

• Tracking Multiple Moving Targets with a Mobile Robot using Particle Filters and Statistical Data Association, by Dirk Schulz, Wolfram Burgard, Dieter Fox, and Armin B. Cremers, 2000

• Using EM to Learn Motion Behaviors of Persons with Mobile Robots by Maren Bennewitz, Wolfram Burgard, and Sebastian Thrun. In: Proceedings of the International Conference on Intelligent Robots and Systems (IROS), 2002.
• Learning Motion Patterns of Persons for Mobile Service Robots by Maren Bennewitz, Wolfram Burgard, and Sebastian Thrun. In: Proceedings of the International Conference on Robotics and Automation (ICRA), 2002.