Colorizing the Prokudin-Gorskii Photo Collection
Overview
The images in the Prokudin-Gorskii collection consist of a series of 3 vertical panels, each of the same scene, but taken with different filters in front of the camera lens. Conveniently, the images are filtered with blue, green, and red filters (from top to bottom in a panel), which allows us to combine the panels into a single, full-color image. To accomplish this, the individual panels need to be aligned.
Alignment
The panels of an image are aligned to the blue channel using Normalized Cross Correlation (NCC) on an image pyramid. By default, a starting search window of 15 pixels is used, decreasing in size as the algorithm moves down the eight layers of the pyramid. Both the search window and the number of pyramid levels can be changed when the function is called. Pyramid level 0 is the full size image, with image size decreasing by factors of 2 as level increases.
When operating on small images, the pyramid level should be set rather low, probably between 2 and 4, as there are certain images that are dramatically misaligned when colorized with higher pyramid levels. I suspect this is because too much information is lost in the resize to accurately calculate alignment vectors at high pyramid levels. When the alignment vectors are scaled up for lower levels, the images fall out of alignment but the decreasing window size does not allow the proper alignment to be found.
I found that the amount the window changed between levels had a large effect on the alignment of the images. Simply dividing by 2, to match the scaling factor proved to reduce window size to quickly. In the end, a solution which reduces widow size based on the current level and the total number of pyramid levels was used. This seems to work well for most images with the default starting values for the window and number of levels (15 and 8). However as mentioned above, there are some images where the number of levels or the starting window must be changed &mdash normally decresed. I suspect this is because these images are mostly aligned to begin with; allowing the search to range too far away from an offset of (0, 0) causes similar features in different parts of the image to "match", resulting in a falsly high correlation.
Images
Low-Res Images
Default Settings
Default Settings
Window = 8, Level = 2
Result with default settings.
Window = 15, Level = 4
Result with default settings.
I belive this image does not align very well because there are no strong edges or highly unique regions. The sky takes up a large part of the image and is mostly the same between color channels and the trees have many fine features instead of serveral large, defined features. Also, in the blue channel, the tops of the trees are overexposed, compared to the other two channels.
Default Settings
Default Settings
Default Settings
Window = 15, Level = 5
Result with default settings.
Window = 15, Level = 3
Result with default settings.
Default Settings
High-Res Images
Note: All TIFs were converted to high-quality JPEGs at the same resolution for processing, to save AFS disk space. The results are presented here as full resolution JPEGs
Default Settings
Default Settings
Default Settings
Default Settings
Default Settings
Default Settings
Default Settings
High-Res Images from Library of Congress
Default Settings
Window = 8, Level = 4
Default Settings