Beautiful Color

Here are some select sections of the colorized images that aligned very well, and had general good color quality.

 


Full Results

Scalled down resulting images from all required images, and two images personally chosen from the full collection from the Library of Congress. Notice that two images did not align. One of these images is that of the train. My simple alignment cost function, described further down, did not handle the large difference in color channel intensities that were present in this image. The other image is that of a woman. The reason for this misalignment is due to the differences in the three channel image shots. The subject most likely moved too much between the three exposures.



Colorizing Process

The basic process used to colorize the Sergei Mikhailovich Prokudin-Gorskii images is described below. Most of the steps are rather straightforward and simple except the "alignment" step. The three separate color channel images must be aligned correctly before they are combined. Different alignment approaches may be used. However, I choose one of the simplest methods that yielded reasonable resulsts.

Red
Green
Blue
(0) Full original image. (1) Sobel edge filter. (2) Borderless image based on sobel edge information. (3) Separate blue, green, and red channels. Original image split into 3 pieces. (4) Recursively align one channel to another. A pyramid of images sizes is used for more efficient alignement searching.

When aligning channels, you assume one to be correct, then align the other two independently to the "target." For my implementation, the alignment algorithm was fairly simple. I used a simple Sum of Squares Difference (SSD) as a "cost" function between the two images. Recursively I shift the image around and find the minimum cost shift. While returning up, the search window is halved. Once this step is complete, you simply merge the three color channels together to form a full color image. I do not crop the resulting image at all, so the borders have artifacts of the images not being fully aligned. However, the results still look decent:

(5) Final result with all 3 channels merged. The alignment is not perfect, but reasonable. The image is also slightly blurred due to the alignments not matching up perfectly. Some post-processing could help.

Difficulties

Even the simplest alignment shift search yields decent results. However, the white borders (artifacts of the scanning process) around the top, bottom, left, and right of the original scans throw the alignment process way off. This is due to the fact that the top (blue) and bottom (red) images will have less "image" and more border when we naively split the image into 3 equal pieces. If you simply eliminate this top and bottom border the simple equal 3-division works alright. However, one of the best ways to improve the alignment process is to have the channel images contain primarily "image" and less extra border. Even taking this into account, two images did not align properly. The reason for this is explained above in the results section.

Bells & Whistles

The only real bells & whistles implemented is the border cropping at the beginning of the colorizing process. I also applied a sharpening ("unsharp") filter at the end to counter-act some of the blurring.