MPI-Video Surveillance and Monitoring:
Implementation and Results

We implemented the MPI-Video EM using C++. The implementation consists of:

We have also implemented a small set of layers that demonstrate the operation of our MPI-Video infrastructure.

How to Build an MPI-Video Environment Model

To build an MPI-video system, one must first decide what layers are needed and then implement them. This is done by creating subclasses of the EM layer base class. The subclass must contain state variables and methods to override the base class initialization, extrapolation and update methods. Most details of the iterative system are hidden from the implementation of a single layer type so these methods tend to be easy to write, however, a working knowledge of C++ is required. A layer need not always take advantage of the dynamic capabilities of the system. When there is no reasonable dynamic model of the state of a system, e.g., image data and static environment information, we simply use an identity operation for the extrapolation. A layer must provide a means for other layers to access its state. When a layer does its update, it is provided with a list of pointers to layers from which it obtains input. The update method invokes the state access methods for the other layers to get their data, and then performs an update using that data. Once the subclass for a layer type is defined, it is added to the library of all layer types.

With a sufficient library of layer types in hand, it is then necessary to configure the EM. We do this with a configuration file (plain text) that describes the layers in the EM and how they should interact. The file consists of a list of layers in the EM. Each layer has a layer type, a name, a group name, some initialization parameters, and a list of other layers from which it receives input. The group name is required for keeping track of groups of layers that work together. For example, when a three-dimensional tracking layer combines data from multiple cameras, it also needs camera calibration data from multiple layers and it must match each set of calibration data with the correct camera. Group names resolve the ambiguity. An example configuration file follows.

# EMV1.0
# Spec file to show EM for algorithm evaluation. Segmentation routine
# running w/ different parameters on same input sequence.

# column contents
# type | name | group |input connections | init info;

# Same input stream.
EmMpiv0In   vidIn0   group0            {"twocam0.mpiv"};
EmSegment   seg0     group0 vidIn0     {"initAlgorEval1.seg"};
EmSegment   seg1     group0 vidIn0     {"initAlgorEval2.seg"};
EmViz       vizlayer group0 seg0 seg1  {"init2.viz"};

Example of an MPI-video EM configuration file.

The last step in building an MPI-Video system is to create a main program that invokes a method to build the EM from the configuration file, and then invokes a method to run the EM. To date, we have only used a single main program and alter the EM by changing the code for layers and by editing configuration files. Once a sufficiently powerful library of layers is developed over time only the configuration files will need to be altered to create new EMs.

At the time of writing, we have not implemented access methods to facilitate server/client interaction. Therefore, in order to visualize the state of the EM and verify its operation, we implemented visualization layers. These layers have no state and do no extrapolation. They simply display the image data that they get from other layers during update. In the long term, visualization will be done by an EM client, but visualization layers allow for easy debugging of a system without the need for a client.

A Running Environment Model

The figure below shows a screen snap shot taken during the operation of a simple MPI-video system.

Screen snap shot of an MPI-video system in operation: (bottom) video stream from file, (upper left) segmentation, bounding box and centroid of person in scene, (upper right) alternative segmentation of video stream.

Before running the system, we placed a set of four cameras in the corners of our MPI-Video test bed, and recorded the video signals digitally for about 60s while a person moved about the room. We then ran the system using and EM with layers to read recorded video data from files, layers to segment the images, and visualization layers to show both the raw video data and the segmented images. For presentation on a printed page, the snap shot shows only the images for one camera view at a single instant in time. We observed the following during our development of the EM:

Source Code Availability

We plan to make this infrastructure available to the public, early in 1998. The first distribution will include the EM layer base and EM state base classes, an initial set of layer types, and working examples of configurations based on those types.

MPI VSAM Home Page -- Executive Summary -- Our Vision -- Project Information --
Papers -- Investigators -- Administrative Staff -- Related Links

Web page design by Jeffrey E. Boyd