Image Mosaics
I did these in 1995 for a class on rendering.
By computing the projective mapping between the two images,
then warping image 2 to match image 1's projection, the images are joined
together to form a composite image. To do this, four pairs of matching points are required for each pair of images. Initially I specified the matching
points manually. Later I extended the program to search for corresponding
points ("pseudo-features", since they were often not points that a person
would consider to be features of the images.)
All the image sets were photographed in Squirrel Hill, the neighborhood
where I lived in Pittsburgh.
Source Images & Mosaics
Some source images show markers (small red dots) indicated the
correspondence points that were used to generate the projective
transformation. These markers are NOT part of the original input
image. They were computed during the mosaic creation, and added to
the images for display purposes.
Many mosaic images have annotations "(SSD m, win n)". This means that
(pseudo-)features were refined by searching within m pixels of the
original location, using an SSD window of nxn pixels.
Thanks to everyone who gave me their images to test this program.
Only a few are shown here.
Simple Examples
Apartment Houses
Image 1
Image 2
Mosaic
The Jewish Community Center
Image 1
Image 2
Mosaic
Church Tower
Source Image 1
Source Image 2
Automatically produced mosaic with L-M refinement (SSD 5, win 15).
The odd dark blue triangle in the sky just over the top right corner
of the blue & white building shows where the corner of the lower,
slightly darker image lies in the composite.
Automatically produced mosaic (SSD 5, window 15, without L-M refinement))
Bad mosaic with Levenberg-Marquardt refinement. The automated process
correctly matched up three points (NW, SW, and SE corners of overlap
region) but missed the NE corner. Hence the blurriness at the top right
of the blue & white building but fairly clear compositing elsewhere.
Church (with markers)
Automatically produced mosaic with Levenberg-Marquardt refinement. This
example shows how much one image may have to be warped to meet the other.
The very small overlap of the two source images makes the mosaic very
sensitive to small registration mistakes.
This mosaic used manually-selected correspondence points (SSD 5, win
5). The improvement is most obvious in the big arched window in the
middle section of the building, which is right in the middle of the
overlap region.
JCC
This mosaic was made early in the program's development, and shows
some of the things that can go wrong. The difference in scene
illumination confused the mean-square-error brightness metric into
putting the SW marker on pixels of two different colors. This mosaic
was created before the program was modified to move the
pseudo-features inward until they found regions with significant
variation. As result, the NW and NE pixels are in regions where a
significant registration error won't be noticed. Also, the blending
function was later improved so that visible seams (such as the
vertical line running through the yellow bricks just to the right of
the flagpole) don't occur.
Source image 1
Source image 2
Mosaic (SSD 25, win 15, with L-M refinement).
Mosaic computed from manually-specified correspondence points,
with L-M refinement.
Houses
The mosaics were produced from a file of manually-identified
correspondence points, to which some noise (up to 5 pixels error) was
added. This demonstrates that Levenberg-Marquardt refinement doesn't
make all that much difference compared to SSD correspondence-point
refinement.
Source image 1
Source image 2
Mosaic without L-M or correspondence-point refinement
Mosaic with L-M, no correspondence-point refinement. The top left corner of the near house looks WORSE, although the windows are clearer.
Mosaic with correspondence-point refinement, no L-M.
One more view of the houses: an automatically generated mosaic using 4 pyramid
levels and L-M refinement. (SSD 5, win 5)
Back to my home page
Ari Rapkin
Last modified: Thu Oct 17 20:43:33 EDT 1996
