Project 1: amihalci

The goal of this project is to build color images from red, green, and blue exposures taken by Sergei Mikhailovich Prokudin-Gorskii at the beginning of the 20th century. The full set of original images is available at the Library of Congress website.

To align the red, green, and blue channels, I used a multiscale pyramid implementation. I used either 4 or 7 pyramid levels, depending on the size of the image (more on that later). Each time I aligned two channels (blue with green, then blue with red), I built a pyramid of both images, and successively aligned each corresponding level of the two pyramids.

The image at pyramid level i + 1 is twice the width and twice the height of the image at pyramid level i. The bottom image in the pyramid is actually the given image, so we are computing the pyramid from bottom to top here. As suggested in the starter code, I used the imresize function to resize the image without introducing aliasing.

Starting at the top level, I computed the best alignment between the corresponding levels of the pyramids. For all levels but the top, I started with the same offset computed at the previous level, adjusted for scale. At the top level, I assumed that the previous level's offset - which of course didn't exist - was (0, 0). Then at each level I searched for the best alignment between the two images in a 7 pixel by 7 pixel square centered on the previous level's offset. For each shift in that 7 by 7 square, I just computed the distance between each shifted version of the images using the SSD metric, and picked the offset that caused the lowest sum of squared distances.

I mentioned earlier that I used a 4-level pyramid for small images and a 7-level pyramid for larger images. The reason for this is that, in this particular collection of images, larger images are higher-resolution scans of the original photographic plates. This means that a constant alignment issue in inches of photographic plate corresponds to more pixels in the higher-resolution images. Therefore, we need more levels in the pyramid for larger images, in order to allow for larger pixel distances between colors. There is a second reason to have fewer levels for small images: if a level of the pyramid gets too small, the SSD metric is roughly the same for all alignments. This means that we may get a large offset at the top of the pyramid and never recover from that large offset at the lower levels of the pyramid. In fact, I originally used a 7-level pyramid for all images, but the image 00757v.jpg didn't align properly for the reason I just described. Once I switched to the strategy of different number of levels for different image sizes, this problem disappeared.

Bells and Whistles

To automate some of the human work involved in restoring an image, I wrote a routine to trim the bad edges off of an image. This function, called trim_image, uses the built-in Matlab edge function to find horizontal and vertical edges, and then trims at the longest edge in the outer 5% of the image. The reasoning behind this is simple: where the colors don't line up or are exposed differently, there will be a lot of differences between colors, generating vertical edges on the left and right sides and horizontal edges on the top and bottom sides of the image.

To implement this, I tried a couple of different approaches. My first approach was to look for edges in the average of the three colors. If there are large differences between the colors in one region, that will cause edges in the average image. This approach worked reasonably well, but not as well as I'd hoped. My final approach was to look for edges in the average, and in each of the three colors separately, and take the most-cropped image from all of those. This approach seems to work well. It does run into problems because many of these images have a white border on one side due to the scanning of photographic plates. For this reason, the longest image edge is sometimes the boundary of the photographic plate itself, not the boundary between good color and splotches of bad color.

Image 00998 - Before

Image 00998 - After

The Enhanced Images

Course-Supplied Images

Image 00029

Green Offsets: (38, 18)
Red Offsets: (90, 36)

Image 00087

Green Offsets: (48, 40)
Red Offsets: (109, 56)

Image 00106

Green Offsets: (4, 1)
Red Offsets: (9, -1)

Image 00128

Green Offsets: (35, 25)
Red Offsets: (52, 37)

Image 00458

Green Offsets: (42, 6)
Red Offsets: (86, 32)

Image 00737

Green Offsets: (15, 5)
Red Offsets: (49, 14)

Image 00757

Green Offsets: (2, 3)
Red Offsets: (5, 5)

Image 00822

Green Offsets: (57, 25)
Red Offsets: (125, 34)

Image 00888

Green Offsets: (6, 1)
Red Offsets: (12, 0)

Image 00889

Green Offsets: (1, 2)
Red Offsets: (4, 3)

Image 00892

Green Offsets: (16, 3)
Red Offsets: (42, 5)

Image 00907

Green Offsets: (2, 0)
Red Offsets: (6, 0)

Image 00911

Green Offsets: (1, -1)
Red Offsets: (13, -1)

Image 01031

Green Offsets: (1, 1)
Red Offsets: (4, 2)

Image 01043

Green Offsets: (-15, 10)
Red Offsets: (11, 18)

Image 01047

Green Offsets: (24, 19)
Red Offsets: (71, 33)

Image 01657

Green Offsets: (5, 1)
Red Offsets: (11, 1)

Image 01880

Green Offsets: (6, 2)
Red Offsets: (14, 4)

Images I Downloaded

Image 00564

Green Offsets: (52, 5)
Red Offsets: (112, 1)

Image 00998

Green Offsets: (18, 16)
Red Offsets: (58, 18)

Image 01033

Green Offsets: (19, 29)
Red Offsets: (42, 36)

Image 01070

Green Offsets: (52, 46)
Red Offsets: (125, 85)