Project4: Image Warping and Mosaicing

Danielle Millett

This project was to warp images to appear as if they were taken from a different angle. The warp is done by applying a 3x3 matrix called a homography to the image. To find this homography you need to have 2 sets of corresponding points in your original image and your resulting image.

Image Rectification

Using this homography, you're able to do image rectification and change the perspective on an image. To do this I found 4 corners of a square object in the image, and then warped these to be a square. This new image has this surface as being parallel to the camera. Here are some of my results:

Original Image

Warped Image


Homographies can also be used to create a panorama. First I picked corresponding points between two images and used them to find a homography warping one image into the other. This results in two images that are from the same perspective that can be pieced together into one image. To piece them together I used the alpha channel and for each image, feathered the alpha center from 1 in the center of the image to 0 to the outside of the image, leaving a square in the center that was all 1's. After overlaying all the images, I took a sum of all the alpha channels and divided each one by it so that the total alpha value would always be 1. This creates a smooth blending between the two images when you overlay them. Here are a few examples:

Auto-stitching Panorama

Having to manually enter corresponding points between panoramas is very time consuming, so we want to find a way to automatically determine corresponding points. There are several steps involved in doing this:
1. Use harris corner detection to determine possible feature points.
2. Use adaptive non-maximal suppression to pick a subset of these points that are evenly distributed over the image.
3. Extract a feature descriptor for each point. This is an 8x8 patch that was sample over a 40x40 window. It is also normalized to account for differences in bias/gain.
4. Find matches between the features by finding the top two nearest neighbors. Using the ratio between the first and second nearest neighbor, you can set a threshold that will give a subset of reasonable matches.
5. To further weed out outliers in matches, RANSAC is performed, which computes homographies from random points and then looks to see how many of the matches correspond with this homography warp.
Harris Corner Detection Adaptive non-maximal suppression
Feature matching before RANSAC
Red and Blue are before RANSAC, Red is after RANAC
Final Panorama using autostitching

Bells and Whistles

For stitching together the panoramas, I was using alpha channels to blend the images, which removed any lines between the images, but if there was a lot of overlap, it created some bluring and ghosting effects. So to remove these effects I implemented min cut between the two images. Because this sometimes created lines if the images weren't exactly the same, I did a gaussian blur on the min cut so that it would blend them a little. Here's the alpha channel corresponding to the images in this panorama:

The results for these generally turn out better given that the two images are similar in light conditions. Here is the result for the same image as above, but using min cut instead. Notice that the chair on the right and in the center aren't blurred anymore.

Here are other results from panoramas using min cut:

Although min cut works fairly well on most images, it does give artifacts sometimes if the images don't quite line up. You can see in the following one that the windows on the right didn't quite line up.

Image overlaying

Another thing I did was used the homography warping to overlay an image onto another. This was done by taking the one image and warping it to the perspective in the second image. Then overlaying them and blending them. Here is an example where I overlayed tartans in the football field end zone: