15-463 Project #4: Image Warping

Done by: Junjie Liang (junjieli)

12 November 2010

Overview

In this project, we were required to generate photo mosaics, or what is commonly known as a panorama. We wrote two functions:
computeH(im1_pts, im2_pts);
that takes in correspondence points im1_pts and im2_pts between two images, and produces the homography matrix for the transform. We also had a function that takes in a homography matrix and applies it to a given image.
warpedim = warpImage(im, H);

Procedure

For this part of the project, we defined correspondence points manually by hand. I used cpselect from matlab which gave a nice interface to define the correspondence points. Of particular interests were corners, which were easy to match by eye, and allowed for very accurate point to point correspondence.

After that, it was a straightforward matter of computing the homography matrix, and warping the images accordingly. Image blending (for the mosaics) was done with a 2-level pyramid, which was sufficient for nice results.

Results - Rectifying an image

Original:

Rectified:

Original:

Rectified:

Results - image mosaic

Manual Mosaic 1: Source images
   

Manual Mosaic 2: Source images
   

Manual Mosaic 3: Source images
    

Part 2 - Automatic image stitching

In this part of the project, we stitch images together to form mosaics (as before), but this time the stitching was to be done automatically. In my experience, this was a much better approach, because the results were as good as, if not better, than the manual correspondence points, and we skip the troublesome step of manually determining correspondence points.

Automatic - procedure

We implemented some of the ideas in "Multi-Image Matching using Multi-Scale Oriented Patches" by Brown et al., but with much simplifications. Surprisingly, even with the simplifications, the results were very good. First, given two images we determine the corners using a Harris Interest Point Detector.

Then, we extract the 500 most interesting points from each image, using the Adaptive Non-Maximal Surpression algorithm explained in the paper.

Then, we pick the points that have good matches in the other images. A good match is determined by looking at a 8x8 patch around each point, and checking to see if a similar patch exists in the other picture.

Lastly, we run RANSAC 1000 times to find a good set of points, from which we can compute our homography matrix H.

To summarize, here are the points superimposed on the same image, where red = top 500 interesting points, blue = interesting points with matches, yellow = points selected through RANSAC.

Results

Mosaic 1: Source images
   

Mosaic 2: Source images
     

Mosaic 3: Source images
     

Mosaic 4: Source images
    

Mosaic 5: Source images
      

Bells and whistles

We can warp images and place them at different places. Here's a hole through a door that leads to the Big Ben:

And we can play around with the morph from the previous project.