Smart Projectors: Camera-Projector Systems

Shameless plug: IEEE International Workshop on Projector-Camera Systems (PROCAMS)

Introduction

This project explores synergies between cameras and projectors. We are actively working in the following areas:

Automatic Keystone Correction

 Current approaches to keystone correction, whether optical or digital, only address the limited class of distortions caused by vertical misalignment of the projector symmetric trapezoidal keystoning, and require manual adjustment. Our system corrects all distortions due to misaligned projector placement, without need for human intervention. This is done by prewarping the image that is sent to the projector, in such a manner that the prewarping precisely negates the distortion caused by projector misalignment. One cannot solve the problem simply by prewarping the image so that it appears undistorted to the camera since the camera is itself not aligned to the presentation screen.

These images provide some insight into the problem, and our solution is fully-described in the following papers.

The following patents have been filed by Just Research on this technology:

Vision-based Presentation Control

 Vision-based presentation control frees the speaker from standing beside the computer while delivering a talk. The speaker uses a pointing device (typically a laser pointer) to activate virtual buttons on the projected slide to drive the presentation. The system watches the presentation screen through a camera placed anywhere in the room (much like a human audience member). The position of the pointing device in the camera image can be determined using image differencing (or any other standard computer vision technique) and the mapping between the camera image and the speaker's slide is given by the projective transform described in the calibration section. The camera-assisted presentation system also allows the speaker to draw on the slide using pointer gestures, either to highlight specific points, or to make virtual annotations. The system captures low-resolution (160x120) images from a low-cost digital camera at 20Hz, and (using super-resolution tricks) achieves a tracking error of +/- 3 pixels on a 1024x768 slide. For details, see: The following patent has been filed by JustResearch on this technology:

Dynamic Shadow Elimination for Multi-Projector Displays

 We have developed a new application for camera-projector systems where multiple front projectors are used to generate redundant illumination over the display surface. A multi-projector display with shadow elimination could provide a good alternative to expensive rear-projection systems that require specialized projection surfaces and space behind the screen for projectors. The projectors are placed at extreme angles but oriented so that their projection areas overlap significantly. By appropriately pre-warping the images sent to each projector, the system generates a sharp, keystone-corrected image in the overlap zone. Redundant illumination makes the display resistant to occlusions: the content in a partially-occluded region is readable as long as one projector maintains an unblocked light path. Unfortunately, the occlusion still causes a shadow in that region (visible as a darker patch). We demonstrate a system that automatically detects and dynamically eliminates these shadows so that the display surface appears shadow-free even in the presence of multiple, moving occluders. The system dynamically identifies occlusions using cameras, and eliminates shadows by appropriately adjusting the images projected by each projector. Rather than locating occluders by tracking objects in the environment, our approach focuses exclusively on detecting artifacts on the display surface. Shadows are eliminated using a feedback loop that requires no explicit photometric models of the environment. No assumptions are made about the locations, sizes or shapes of occluders.

For details, see:

Occluder Light Suppression

We have extended our shadow elimination system to simultaneously remove the light falling on occluded objects.

For details, see:

Scalable Alignment of Large Multi-Projector Displays

 We present a practical vision-based calibration system for large format multi-projector displays. A spanning tree of homographies, automatically constructed from several camera images, accurately registers arbitrarily-mounted projectors to a global reference frame. Experiments on the 18'x8' Princeton Display Wall (a 24 projector array with 6000x3000 resolution) demonstrate that our algorithm achieves sub-pixel accuracy even on large display surfaces. A direct comparison with the previous best algorithm shows that our technique is significantly more accurate, requires far fewer camera images, and runs faster by an order of magnitude.

For details, see:

Rahul Sukthankar (rahuls@cs.cmu.edu),