Colouring the Prokudin-Gorskii Photos

Tian Jin, 15-463 Assignment 1, Fall '15

In this assignment, I attemted to take the Prokudin-Gorskii photos and align the three channel images to produce a coloured RGB image. The approach was rather straightforward: I separated the image into the three parts, Red, Green and Blue (R, G, B). Then, I took the middle 80% of each channel by removing 10% of the rows and columns from each side. Using these smaller matrices, I then attempted to align R and G to B. To do this, I would first shift the matrix by a displacement vector (x,y), where x and y lie in the range [-15, 15]. Then, I computed the Sum of Squared Distances for the shifted matrix and B. After trying all values in the range, I took the displacement that produced the smallest SSD value. Taking these displacements, I was then able to align the three images to form a RGB colour representation.

For larger images, recursion was used to make the algorithm more efficient. I would recursively call the align function on a matrix of half the size, until the images were quite small. I then aligned the small images, and then passed up the displacement. The displacement from the small image, was then used as the focal point to align the larger image. So instead of searching a 30 by 30 window in the larger images, I would use the displacement found at the previous level of recursion and search only a 6 by 6 window around the displacement. By narrowing down a point on a smaller version of the image, I was able to search less displacements on the larger images. A problem I ran into doing this was the fact that I forgot to multiply the displacements from the smaller image by 2 so that they were pointing to the correct pixels in the large images. This caused my algorithm to focus around the wrong points until I fixed this error. For larger images, I used 4 levels of recursion.

I also attempted a cropping algorithm to remove the borders of coloured images. I achieved some success on the small images, though it did not always work on the large images. If I had more time, I would look for better ways of detecting a solid shade on the edges of the image, as right now I'm merely looking for a solid column or row of extreme intensities.