15-463: Computational Photography
Programming Project #1: Colorizing The Prokudin-Gorskii Photo Collection

Aaron Lo

Basics:
I started with the basic alignment algorithm using SSD with [-15 15] window and most of the small JPG images worked well. So the first thought came to me are the bad edges and they are easy to get rid of. I just need to crop 60% of the original image from the center.

Multiscale:
After done with the JPGs, implementing the multiscale version is pretty straigt forward but then some issues came up. First problem is performance. It's already pretty slow to run a [-15 15] window on the scale of a [1000 1000] image don't even mention the original images are about [3700 3200]. But we don't really need a constant window because on a bigger scale we already have a pre-aligned image and just need more fine tuning so I scale down the window to 80% on every scale till a minimum of [-5 5] which has a pretty reasonable processing time even on the final scale. I did think about just sampling say just four corners and center of the image with a smaller window but since [-5 5] window is fast enough, I didn't bother to implement that.

Final problem:
After testing a couple of the TIFs with success I wrote a batch script to process all the example images. Everything turned out perfectly except this one single 00153u.tif. I spent two days trying to figure out if there's bug in my code but despite all the efforts changing my algorithm it always seem to shift alway to the right. So I look up for the original image and found out it's a more fundamental problem. Because I always use the red channel as target, this man wearing the blue clothes appears to be completely blank in the red channel and that is causing the SSD to not work properly. So I simply reorganize the channels and use the green channel as target and it solves the problem.

Bells and whitles:
Edge cropping is just as described on the assignment page. Just need to play around with the threshold. Also it's very easy to implement the auto color and auto level filter and there are even matlab source code available on Mathworks website. I did try them out but didn't make much of a difference so I didn't include the results.

Images:

00017v.jpg G disp [0 1] B disp [-1 3]	00033u.tif G disp [-51 -3] B disp [-104 -15]
00056v.jpg G disp [-7 1] B disp [-12 -1]	00084v.jpg G disp [-7 5] B disp [-11 9]
*00153u.tif R disp [72 15] B disp [-69 -25]	00362v.jpg G disp [-5 5] B disp [-10 5]
00498v.jpg G disp [-5 -1] B disp [-10 -3]	00646v.jpg G disp [-8 -1] B disp [-15 -4]
00704v.jpg G disp [-7 -1] B disp [-13 -3]	00794u.tif G disp [-70 -2] B disp [-127 -17]
00797u.tif G disp [-77 -1] B disp [-145 -14]	00858v.jpg G disp [-7 -1] B disp [-13 -4]
00872v.jpg G disp [-1 1] B disp [2 2]	01039v.jpg G disp [-6 0] B disp [-11 -2]
01443u.tif G disp [-46 -23] B disp [-80 -44]	01728v.jpg G disp [-10 0] B disp [-18 -1]
01754u.tif G disp [-56 -31] B disp [-92 -21]	31421v.jpg G disp [-5 0] B disp [-14 0]