Computational Photography

Project 1 – Image Alignment

Yong He

09/02/2012

 

Description

Write a program in MATLAB that reads a grayscale image consists of RGB parts of same scenery. Divide the image into separate RGB parts and align them together to create an RGB colored image.

My Approach

The core problem in this project is to find a proper algorithm to align three different parts, a.k.a. find out a coordinate offset in x, y axis for R channel and G channel image so that they can be aligned to B channel after being shifted. The easiest way is to brute-force search for such coordinates: at each possible coordinate, evaluate its score through an error estimator and pick up the coordinate with best score. I tried both SSD and NCC estimator and NCC turns out better.

To handle large images, recursive searching (i.e. traversing the image pyramid) is implemented. However the outcome of recursive searching is far from satisfactory. For most images, recursive method could result in errors as large as 20 pixels. One possible reason is that as the algorithm traverse to upper layers of a hierarchy, image has to be down-sampled and details would be lost. Therefore an optimal choice at upper level does not necessarily contain the best choice at finest level.

In order to obtain better results, I also tried to apply a sobel/laplacian filter in hope that the algorithm could align the images according to edges. However as intensity of the three parts differs, this approach fails. Then I tried to rescale the intensities so that all three parts have the same average intensity. However this does not help.

In the end I tried to address the problem in frequency space rather than in spacial space. The trick is to compute the phase correlation between the fourier transform of each image, and find the position of the peak in the phase correlation. Use the peak position as the offset coordinate will align the phases in frequency space. This approach turns out to be very fast even for large images, and works much better than intensity based alignment methods. To crop the border of a restored image, I exploit the fact that the all the borders are in darker color with less variance. The algorithm is to go through each row and each column at 0%-16% positions in the image and check if the average intensity of any channel in that row or column drops below a threshold, then use the position as crop border.

Results

The offset coordinate of each image are notated as G(x,y) R(x,y), representing the offset coordinate of G channel and R channel respectively.


00029u.tif

G(13,42) R(34,92)

00087u.tif

G(37,55) R(57,101)

00088v.jpg
G(4,4) R(6,5)

00106v.jpg
G(2,5) R(401,10)

00128u.tif

G(27,40) R(37,50)

00137v.jpg
G(7,7) R(10,12)

00737u.tif

G(9,18) R(15,51)

00757v.jpg
G(4,3) R(6,6)>

00822u.tif

G(25,60) R(35,123)

00888v.jpg
G(2,7) R(2,13)

00889v.jpg
G(3,3) R(4,5)

00892u.tif

G(3,17) R(4,41)

00907v.jpg
G(1,3) R(1,7)

00911v.jpg
G(395,2) R(395,14)

00992u.tif

G(12,52) R(20,114)

01031v.jpg
G(2,2) R(3,5)

01043u.tif

G(14,3194) R(15,14)

01085u.tif

G(33,45) R(60,114)

01734u.tif

G(29,49) R(49,102)

01880v.jpg
G(3,7) R(5,15)

Other Images


00548v.jpg G(2,8) R(1,15)


01115u.tif G(2,8) R(1,15)

Comparison between Spacial space and frequency space aligning methods

Spacial space method Frequency space method

Border Cropping: before and after

Before After