Vision-Based Obstacle Detection for the Toro Riding Lawnmower


The goal of the Toro project is to automate a Toro riding lawnmower, for use on the fairways of golf courses. The challanges in this include navigation, coverage, and perhaps most importantly, given the platform, obstacle detection.

Various methods of obstacle detection are being investigated, including radar, laser, and computer vision. A pure computer vision solution would be ideal, as vision systems are generally much cheaper than radar and laser sensors. However, there are a number of challanges to using only vision, including:

The first approach is color segmentation. Grass is relatively easy to segment out, and up to a point, shadows can be dealt with by modeling the contribution of sunlight to the PSD of the un-shadowed regions, and compensating. The simplest segmentation is to find the median color in the image, and classify anything color a set distance from the median color as an obstacle. This assumes that the majority of the image is non-obstacle. This works reasonably well, as the images and movies below show.

However, a slightly more sophisticated approach yields better results. By using a training set of non-obstacle pixels, we can construct a PDF of the expected colors for grass. Then, when testing on new images, we determine whether the pixel is grass or not by thresholding on it's probability of being grass, based on the prior training data. There is a movie of this below.


These images were segmented using a color-threshold approach. Original image:
Input Image

Obstacle image without shadow compensation:
Obstacle image with shadows

Shadow compensated image:
Compensated Image

Obstacle image with shadow compensation:
Obstacle image without shadows

NEW Movies

recFrontLawn.mpg - A 60x timelapse of the mower mowing the REC front lawn area


MowerWalkAround.avi - a quick tour of the mower platform - 12 Megs, AVI format.
toro_navigation.mpg - a 4 minute movie of the mower navigating a coverage pattern while detecting obstacles. 16 Megs, MPG format
Color threshold segmentation (no shadow compensation) - 8 Megs, AVI format.
Color threshold segmentation (shadow compensation) - 10 Megs, AVI format.
PDF-based segmentation - 11 Megs, AVI format.
PDF-based segmentation of a harder sequence - 14 Megs, AVI format.



Parag Batavia, The Robotics Institute, Carnegie Mellon University
Last modified: Tue Nov 13 10:03:42 2001