Heather Strong
15-463 Fall 2008
Project 1: Images of the Russian Empire: Colorizing the Prokudin-Gorskii photo collection

In order to align and colorize the photos I used the Sum of Squared Differences approach and examined the images via an image pyramid. The sum of squared differences worked well in some cases, but terribly in others. In particular, the man in the blue outfit was very difficult to align. The algorithm's basic approach is to take two images and compare them pixel by pixel, summing up the squares of the differences over all pixels. However, with the man in the blue outfit, assume the blue and the red channels were perfectly aligned, and the algorithm was comparing pixels in his outfit. A pixel in the red channel would be very dark, but a pixel in the blue channel very light, so the algorithm would register a large difference between them. However, the images are perfectly aligned, and so we want there to be a very small difference between the pixels. It became clear that this algorithm was not quite doing what I wanted it to.

In order to counteract this, I tried running one of Matlab's edge-finding functions on each channel, then running the Sum of Squared Differences algorithm on the images returned by that function. However, since each of these images was only black and white (not grayscale) and because edges are so sparce, this also yielded very bad results. Then I tried adding the edge image to the original image and running the algorithm on this new, edge-enhanced image. This yielded significantly better results. The added emphasis on the edges caused them to be aligned much better. After this, I implemented a simple color balancing function which would run after the alignment, and a VERY simple cropping function (that simply moves each edge in 10%) to get rid of the junk around the edge of the image. I felt that this yielded fairly nice results.

Regularly Sized Assigned Images

Additional Chosen Images

High Resolution Assigned Images

Bells and Whistles

As mentioned above, there were a total of three enhancements I made to the Sum of Squares algorithm to improve the overall results: Alignment on Edge-Enhanced images, Color Balancing, and Cropping. Below, I have added before and after shots that illustrate the effect that each of these additions had.

Alignment with Edge-Enhanced Images

Before
After

Color Balancing

Before
After

Cropping

Before
After