From Russia with Love

Redeeming Sergei MikhailovichnProkudin-Gorskii

15-463: Computational Photography || Alex Wolfe

A Quick History

Sergei Mikhailovich Prokudin-Gorskii (1863-1944) [Сергей Михайлович Прокудин-Горский, to his Russian friends] wandered around Russia with the Tzar's blessing in order to experiment with color photography. To achieve this, he took three exposures of each scene of his choice, using a red, green and blue filter. He later hoped to combine them using a customized projector and revolutionize the medium, a dream which unfortunately wasn't realized. However, since the Library of Congress has digitalized his efforts and made them publicly available, with the aid of modern technology, we can prevent him from rolling around in his grave in abject depression.

Method to the Madness

The process used to complete Gorskii's vision should be relatively straight foward, in fact he had already worked out the brunt of it himself. Take the three images, which are stored vertically, seperate them, and combine. However, instead of using stacked projectors, we can achieve this digitally, seperating the images based on height, aligning, and compositing.

Gorskii's negatives have held up pretty well considering the their age and the technology used to conceive them. However, their imperfections (the slightly blurred edges, handwritten iteration numbers, a recorded movement unique to the third frame) which make them so beautiful to the naked eye, are akin to horrible wrenches thrown into the calculated perfection of a single algorithm designed to work well for all of them. To overcome this, I do a bit of preprocessing before I calculate the alignment of the photographs. First I use the corner() function in Matlab to distinguish between the clean white border (probably a relic of the scanning) and the distressed black outline to each of the exposures. I then crop based on the smallest and largest corner identified. Afterwards I seperate the three channels into distinct images based on the height of the post-cropped photographs, and then do an extremely rough crop on each of these of about 10% to eliminate as much of the black boarder as I can.

Aligning them happens in a similar fashion to how you would approach the problem if you had the negatives laying on a light board in front of you. Stack, shift one a little bit over to the right, see if it looks better, try a little to the left, and repeat until you find the best results. I acheive this using the Sum of Squared Differences (SSD) to calculate the "cost" of each alignment and doing my best to minimize it. Normalized cross-correlation was another option, but produced virtually indistinguishable results from SSD and was a little slower, so I discarded it. This worked beautifully for the smaller images, using a window of about [-15, 15] to search. However, on the larger images it was god awful. Not only was it horrendously slow, but the results were far from anything remotely resembaling even the distant unattractive cousin of a photograph.

To address this new concern I decided to turn to my good friend recursion. Since the algorithm worked so well on the smaller images, you could recursively create a gaussian pyramid, calculate the best alginment, and pass it up to the next largest image to use as the starting point for the bounds of its window and so on. Only after hours of agony did I realize this was a terible idea. Recursion is not your friend. It is confusing, and in this case (since I'm only doing around 4 pyramid calculations), similar to using Mjölnir to crack a walnut. Completely unecessary. In the end, I simply calculated my four reductions, save them as variables and pass the transform through a slightly modified version of my original alignment function (multiplying the result by two each time I resize the image) without much more code than the recursion method. Also, when I align on the larger images, I only look at the center half of the pixels, to even further rid myself of pesky algorithm smashing smudges/borders.

Results

Small Images

Large Images

Click to view originals (warning: very large files).

R[-16, 17] G[-18, 36]

R[-12, 40] G[-12, 56]

R[-20, 24] G[-58, 37]

R[-22, 4] G[-26, 13]

R[-3, 24] G[4, 33]

R[-8, 3] G[-6, 4]

R[-34, 13] G[-51, 19]

R[-40, 10] G[-39, 18]

R[-10, 31] G[6, 59]

R[-17, 28] G[-28, 50]

R[-12, 6] G[-17, 9]

R[-15, 24] G[-13, 39]

R[-32, 6] G[-28, 12]