Video-Rate Stereo Machine

Video-Rate Stereo Machine


Stereo vision and multi-baseline method

Stereo ranging, which uses correspondence between sets of two or more images for depth measurement, has many advantages. It is passive and it does not emit any radio or light energy. With appropriate imaging geometry, optics, and high-resolution cameras, stereo can produce a dense, precise range image of even distant scenes. Our video-rate stereo machine is based on a new stereo technique which has been developed and tested at CMU over years. It uses multiple images obtained by multiple cameras to produce different baselines in lengths and in directions. The multi-baseline stereo method takes advantage of the redundancy contained in multi-stereo pairs, resulting in a straightforward algorithm which is appropriate for hardware implementation.

Hardware Architecture

The basic theory requires some extensions to allow for parallel, low-cost, high-speed machine implementation. The three major ones are: 1) the use of small integers for image data representation; 2) the use of absolute values instead of squares in the SSD computation for image matching (SAD instead of SSD); and 3) camera geometry compensation capability. A figure below illustrates the configuration of the prototype system. There are five important subsystems: 1) multi-camera stereo head; 2) multi-image frame grabber; 3) Laplacian of Gaussian (LOG) filtering; 4) parallel computation of SSAD; and 5) subpixel localization of the minimum of the SSAD and its uncertainty estimation in C40 DSP array. The video-rate stereo machine will perform these stages on a stream of image data in a pipeline fashion at video rate, resulting in a disparity map in the C40 DSP array at every 30msec.

Current Performance

It is currently operational at the speed of 30 frames per second with 200W200 image size and 23 pixel disparity range. A table below shows the current performance.

Number of cameras       2 to 6                      
Processing time/pixel   33nsW(disparity range + 2)  
Frame rate              up to 30 frames/sec         
Depth image size        up to 256 W 240             
Disparity search range  up to 60 pixels             
It handles the distance range of 2 to 15m with 8mm lenses. Figures below show two example scenes demonstrating the system's performance. The left side images are intensity images of the scene. The right side images are the corresponding disparity maps. The stereo machine successfully generates dense disparity maps in the ceiling and the wall of the corridor which have few features.

Live examples




Related Web Sites


Originally Created by Hiroshi Kano Updated by Kazuo Oda on Jan 4 1996 (last updated Jul 9 1996)