In order to align and colorize the photos I used the Sum of Squared Differences approach and examined the images via an image pyramid. The sum of squared differences worked well in some cases, but terribly in others. In particular, the man in the blue outfit was very difficult to align. The algorithm's basic approach is to take two images and compare them pixel by pixel, summing up the squares of the differences over all pixels. However, with the man in the blue outfit, assume the blue and the red channels were perfectly aligned, and the algorithm was comparing pixels in his outfit. A pixel in the red channel would be very dark, but a pixel in the blue channel very light, so the algorithm would register a large difference between them. However, the images are perfectly aligned, and so we want there to be a very small difference between the pixels. It became clear that this algorithm was not quite doing what I wanted it to.
In order to counteract this, I tried running one of Matlab's edge-finding functions on each channel, then running the Sum of Squared Differences algorithm on the images returned by that function. However, since each of these images was only black and white (not grayscale) and because edges are so sparce, this also yielded very bad results. Then I tried adding the edge image to the original image and running the algorithm on this new, edge-enhanced image. This yielded significantly better results. The added emphasis on the edges caused them to be aligned much better. After this, I implemented a simple color balancing function which would run after the alignment, and a VERY simple cropping function (that simply moves each edge in 10%) to get rid of the junk around the edge of the image. I felt that this yielded fairly nice results.
|