Bruce E. Flinchbaugh
DARPA Image Understanding Program
Texas Instruments Corporate Research Laboratories
The goal of this project is to develop and demonstrate new video understanding technology for battlefield information systems in rural and urban environments. New technology to be developed in this program will focus on video-specific problems that have not been solved reliably in previous image understanding research, with two key technical objectives:
Develop and demonstrate algorithms for robust moving object detection: Robust moving object detection and tracking is needed to detect significant change (e.g. motion of humans or vehicles) in IR and EO video sequences . This effort will produce the technology needed to reliably detect and track people or vehicles in regions under surveillance, in the presence of changes in temperature and lighting, intermittent occlusion, motion of animals and vegetation, and other sources of confusion. The output of the algorithms will specify position in 3D space and attributes such as facing direction (for humans), direction of movement, and appearance attributes for matching and description.
Develop and demonstrate algorithms for video event description and recognition: This work will develop methods of classifying motions and interactions of objects into specific categories constituting events that are important in the current mission context. TI will develop methods of representing and detecting such things as a person parking a vehicle in a sensitive location and then driving away in a second vehicle.
New capabilities produced in this research will be integrated into TI existing video surveillance architecture. The architecture will provide a graphical interface that supports display of detected objects and events on a map of the region under surveillance.
Robust Moving Object Detection and Tracking
Moving object detection algorithms generally work by comparing the incoming video image to a model, detecting and analyzing deviations from the model, and attributing the difference either to the presence of a moving object or to noise. Our approach to robust detection and tracking is to enhance the models to exploit:
Event Detection and Recognition
Our approach to event recognition is based on the TI Automatic Video Indexing (AVI) system (see Relevant Images), which analyzes security videotape recordings. In this system, object motions and interactions are described by a directed acyclic graph called a motion graph. Each node of the graph is an observation of an object, and is linked to its predecessor and successor in time. Forks and joins in the graph represent complex interactions. For example, if a person enters a scene, puts down an objects and leaves, the graph will contain a chain of nodes representing the person, with a fork node whose successors are the continuation of the person track and a chain of observations of a stationary object.
The generic tracking and event recognition capabilities to be developed in this project are applicable to a wide variety of military and defense intelligence needs, and to related problems such as counter-terrorism, drug interdiction, airport security, and urban policing and crime prevention. We will demonstrate the technologies in the context of a system that allows users to interactively specify monitoring and threat detection tasks, and issues alarms or displays results when instructed to do so. The technologies will enable automation of generic tasks such as:
TI will present live demonstrations of the technologies at selected VSAM and Image Understanding workshops, using a combination of live and taped video data. In the demonstrations, the technologies for moving object detection and event recognition will be demonstrated both in isolation and in the context of an end-to-end monitoring application. In the end-to-end application, the algorithms will be used to detect events or behaviors that have been defined as threats, and to respond to threats by issuing alarms.
Courtney, J. "Automatic video indexing via object motion analysis", Pattern Recognition (in press).
Flinchbaugh, B., and T. Bannon, "Autonomous Scene Monitoring System", Proc. 10th Annual Joint Government-Industry
Security Technology Symposium, American Defense Preparedness Association, June 1994.
Flinchbaugh, B., and T. Olson, "Autonomous Video Surveillance", presented at 25th AIPR Workshop: Emerging
Applications of Computer Vision Washington, DC, October 1996.