When monitoring facilities such as depots,
warehouses or parking lots, sensor placement can be planned in advance
to get maximum usage of limited VSAM resources. However, the battlefield
is a large and constantly shifting piece of real-estate, and it may be
necessary to move sensors around in order to maximize their utility as
the battle unfolds. Airborne sensor platforms directly address this
concern. During this program, the Sarnoff Corporation developed automated
surveillance technology to detect and track individual vehicles from a
moving aircraft, to keep the camera turret fixated on a ground point, and
to multitask an airborne sensor between separate geodetic ground positions.
Airborne Platform
The airborne sensor and computation
packages are mounted on a Britten-Norman Islander twin-engine aircraft
operated by the U.S. Army Night Vision and Electronic Sensors Directorate.
The Islander is equipped with a FLIR Systems Ultra-3000 turret that has
two degrees of freedom (pan/tilt), a Global Positioning System (GPS) for
measuring position, and an Attitude Heading Reference System (AHRS) for
measuring orientation. Video processing is performed using
the Pyramid Vision Technologies PVT-200, a specially designed video processing
engine. Video and symbolic detection results from the Islander are
integrated into the ground-based VSAM testbed system via microwave transmissions
captured by a large receiving dish on the ground.
Object detection and tracking is a difficult
problem from a moving sensor platform, since it is hard to detect small
blocks of independently moving pixels when the whole image is shifting
due to self-motion. The key to success with the airborne sensor is characterization
and removal of self-motion from the video sequence using the Pyramid Vision
Technologies PVT-200 real-time video processor system. As new video frames
stream in, the PVT processor registers and warps each new frame to a chosen
reference image, resulting in a cancelation of pixel movement, and leading
to a "stabilized" display that appears motionless for several seconds.
Object detection and tracking is then performed by applying three-frame
differencing to the stabilized video.
It is well known that human operators
fatigue rapidly when controlling cameras on moving airborne and ground
platforms. This is because they must continually adjust the turret
to keep it locked on a stationary or moving object. Additionally,
the video is continuously moving, reflecting the self-motion of the camera.
The combination of these factors often leads to operator confusion and
nausea. Sarnoff has built image alignment techniques to stabilize
the view from the camera turret and to automate camera control, thereby
significantly reducing the strain on the operator. In particular,
real-time image alignment to a reference mosaic is used to keep the camera
locked on a stationary point in the scene, and to aim the camera at a known
geodetic coordinate. More details can be found in Wixson,
1998.
Occasionally, a single camera resource
must be used to track multiple moving objects, not all of which fit within
a single field of view. This problem is particularly relevant for
high-altitude air platforms that must have a narrow field of view in order
to see ground objects at a reasonable resolution. Sensor multi-tasking
is employed to switch the field of view periodically between two (or more)
target areas that are being monitored. This process is described
in Wixson, 1998.
