Vision-Based Obstacle Detection for the Toro Riding Lawnmower

Description

The goal of the Toro project is to automate a Toro riding lawnmower, for use on the fairways of golf courses. The challanges in this include navigation, coverage, and perhaps most importantly, given the platform, obstacle detection.

Various methods of obstacle detection are being investigated, including radar, laser, and computer vision. A pure computer vision solution would be ideal, as vision systems are generally much cheaper than radar and laser sensors. However, there are a number of challanges to using only vision, including:

Performance - The system cannot miss obstacles
Sensitivity - The system should be able to find obstacles as small as a golf ball, without being overly sensitive to false positives
Robustness - While limiting the domain to golf courses has advantages (grass is usually green, after all), illumination can change drastically due to vehicle position, time of day, the season, and presence of shadows.

The first approach is color segmentation. Grass is relatively easy to segment out, and up to a point, shadows can be dealt with by modeling the contribution of sunlight to the PSD of the un-shadowed regions, and compensating. The simplest segmentation is to find the median color in the image, and classify anything color a set distance from the median color as an obstacle. This assumes that the majority of the image is non-obstacle. This works reasonably well, as the images and movies below show.

However, a slightly more sophisticated approach yields better results. By using a training set of non-obstacle pixels, we can construct a PDF of the expected colors for grass. Then, when testing on new images, we determine whether the pixel is grass or not by thresholding on it's probability of being grass, based on the prior training data. There is a movie of this below.

Images

These images were segmented using a color-threshold approach. Original image:

Obstacle image without shadow compensation:

Shadow compensated image:

Obstacle image with shadow compensation:

NEW Movies

recFrontLawn.mpg - A 60x timelapse of the mower mowing the REC front lawn area

Movies

MowerWalkAround.avi - a quick tour of the mower platform - 12 Megs, AVI format.
toro_navigation.mpg - a 4 minute movie of the mower navigating a coverage pattern while detecting obstacles. 16 Megs, MPG format
Color threshold segmentation (no shadow compensation) - 8 Megs, AVI format.
Color threshold segmentation (shadow compensation) - 10 Megs, AVI format.
PDF-based segmentation - 11 Megs, AVI format.
PDF-based segmentation of a harder sequence - 14 Megs, AVI format.

Publications

Pending.

Parag Batavia, The Robotics Institute, Carnegie Mellon University
parag@ri.cmu.edu

Last modified: Tue Nov 13 10:03:42 2001