Steerable Gaze Control for a
Video-based Virtual Surveillant

Principal Investigator: Chuck Dyer

  Department of Computer Sciences
  University of Wisconsin-Madison
  1210 West Dayton Street
  Madison, WI  53706-1685

  Telephone:    608-262-1965
  Fax:		608-262-9777

Technical Area: Video Surveillance and Monitoring (VSAM)

Technical Staff

Project Goals

The goal of this project is to enable rapid, photorealistic scene visualization of real 3D environments such as urban areas and battlefields. Our approach is image-based in that the input is a set of imagery, and no auxiliary data sources such as terrain data or site models are used. This will encompass two primary types of visualizations. First, scene viewing by moving a virtual camera in order to view a real environment from user-selected viewpoints. Second, scene modification by making virtual scene changes and visualizing their effects from arbitrary viewpoints for visual simulations of real environments.

In order to accomplish this objective, we are developing methods for rapidly combining a set of images of a real environment. Video is used as a source of multiple views, in general taken by multiple cameras that are widely distributed around the environment. Output is a set of images to be viewed by a person or used as input to other image understanding algorithms. For both visualization and further processing we are focused on producing photorealistic images of novel views and smooth sequences of views. Thus the main emphasis is on image appearance, not surface reconstruction or model building.

Military Relevance

The military relevance of this effort includes the following activities for customers such as intelligence analysts, special forces operators, combat engineers, and scientists:

In each of the above tasks the raw sensor data may not be well matched with its intended use. Different tasks require different views of a scene, and so the "optimal" views for a particular task may not have been captured. Also, a sensor may be time-shared for multiple uses in a single mission, e.g., when slewing between multiple targets, and interleaving target tracking with systematic scanning of the environment. For these reasons it is advantageous to synthesize a virtual video tuned to the operator's viewing preferences and task-specific targets and activities, thus enhancing capabilities for monitoring and comprehending areas of interest and assessing objects' dispositions.

Technical Approach

Two approaches are currently being researched. The first is called view morphing and is a method for view interpolation from two or more uncalibrated views. The second is called voxel coloring and is a method for combining many widely-separated views.

View Morphing

Voxel Coloring