Image Mosaics

I did these in 1995 for a class on rendering. By computing the projective mapping between the two images, then warping image 2 to match image 1's projection, the images are joined together to form a composite image. To do this, four pairs of matching points are required for each pair of images. Initially I specified the matching points manually. Later I extended the program to search for corresponding points ("pseudo-features", since they were often not points that a person would consider to be features of the images.)

All the image sets were photographed in Squirrel Hill, the neighborhood where I lived in Pittsburgh.

Source Images & Mosaics

Some source images show markers (small red dots) indicated the correspondence points that were used to generate the projective transformation. These markers are NOT part of the original input image. They were computed during the mosaic creation, and added to the images for display purposes.

Many mosaic images have annotations "(SSD m, win n)". This means that (pseudo-)features were refined by searching within m pixels of the original location, using an SSD window of nxn pixels.

Thanks to everyone who gave me their images to test this program. Only a few are shown here.


Simple Examples

Apartment Houses Image 1 Image 2 Mosaic

The Jewish Community Center Image 1 Image 2 Mosaic


Church Tower

Source Image 1

Source Image 2

Automatically produced mosaic with L-M refinement (SSD 5, win 15). The odd dark blue triangle in the sky just over the top right corner of the blue & white building shows where the corner of the lower, slightly darker image lies in the composite.

Automatically produced mosaic (SSD 5, window 15, without L-M refinement))

Bad mosaic with Levenberg-Marquardt refinement. The automated process correctly matched up three points (NW, SW, and SE corners of overlap region) but missed the NE corner. Hence the blurriness at the top right of the blue & white building but fairly clear compositing elsewhere.

Church (with markers)

Automatically produced mosaic with Levenberg-Marquardt refinement. This example shows how much one image may have to be warped to meet the other. The very small overlap of the two source images makes the mosaic very sensitive to small registration mistakes.

This mosaic used manually-selected correspondence points (SSD 5, win 5). The improvement is most obvious in the big arched window in the middle section of the building, which is right in the middle of the overlap region.


JCC

This mosaic was made early in the program's development, and shows some of the things that can go wrong. The difference in scene illumination confused the mean-square-error brightness metric into putting the SW marker on pixels of two different colors. This mosaic was created before the program was modified to move the pseudo-features inward until they found regions with significant variation. As result, the NW and NE pixels are in regions where a significant registration error won't be noticed. Also, the blending function was later improved so that visible seams (such as the vertical line running through the yellow bricks just to the right of the flagpole) don't occur.

Source image 1

Source image 2

Mosaic (SSD 25, win 15, with L-M refinement).

Mosaic computed from manually-specified correspondence points, with L-M refinement.


Houses

The mosaics were produced from a file of manually-identified correspondence points, to which some noise (up to 5 pixels error) was added. This demonstrates that Levenberg-Marquardt refinement doesn't make all that much difference compared to SSD correspondence-point refinement.

Source image 1

Source image 2

Mosaic without L-M or correspondence-point refinement

Mosaic with L-M, no correspondence-point refinement. The top left corner of the near house looks WORSE, although the windows are clearer.

Mosaic with correspondence-point refinement, no L-M.

One more view of the houses: an automatically generated mosaic using 4 pyramid levels and L-M refinement. (SSD 5, win 5)


Back to my home page

Ari Rapkin
Last modified: Thu Oct 17 20:43:33 EDT 1996