Computation Photography Project 1

By: Timothy Farkas

Project Overview

This project involved colorizing images taken by the Russian photographer Prokudin-Gorskii. The photographs that Prokudin-Gorskii took were black and white; however, he took three photographs of each scene: one with a blue filter in front of the lens, one with a green filter, and one with a red filter. These three photographs can be combined by packing the "blue", "red", and "green" photographs into the blue, red, and green channels of a modern digital image. However, each photograph of a scene is slightly translated relative to the other photographs of the scene, so an automated alignment method needs to be performed before the three images of a scene can be combined.

Project Approach

The "blue", "green", and "red" photographs of a scene were aligned by using the Gaussian pyramid of each image and the normalized cross correlation method of comparison. The algorithm is outlined below.

  1. Before alignment was done, the Gaussian pyramid was constructed for each image.

  2. The offsets of the "green" and "red" images were computed relative to the "blue" image. The offsets were calculated using the following steps.

    1. Initialize the x and y offset accumulators to zero.

    2. Begin with the highest level in the Gaussian pyramid (smallest picture) for the two images you are comparing.

    3. Multiply the x and y offset accumulators by 2. This will do nothing the first time around, but as you traverse down the pyramid the image sizes double, so the offsets that you've calculated so far must double as well.

    4. Get the center patch for the first image.

    5. Get each patch within a 13 by 13 area (or something like that) surrounding the center patch of the second image and compare each patch with the center patch of the first image using the normalized cross correlation.

    6. Save the offsets of the patch in the second image that matched the center patch in the first image the best.

    7. Add the offsets to the x and y offset accumulators mentioned in step a.

    8. Repeat steps 3 through 7 until the bottom of the pyramid is reached. The values in the offset accumulators are the offsets that you need to use.

  3. Manipulate the offsets a bit to make indexing into the images easier.

  4. Combine the three channels by using two for loops.

  5. Done.

Results On Given Data

00029u

Red Channel Displacement: x=-36, y=-91
Green Channel Displacement: x=-19, y=-38
Blue Channel Displacement: x=0, y=0

00087u

Red Channel Displacement: x=-57, y=-108
Green Channel Displacement: x=-40, y=-48
Blue Channel Displacement: x=0, y=0

00088v

Red Channel Displacement: x=-5, y=-4
Green Channel Displacement: x=-3, y=-3
Blue Channel Displacement: x=0, y=0

00106v

Red Channel Displacement: x=0, y=-9
Green Channel Displacement: x=-2, y=-4
Blue Channel Displacement: x=-1, y=0

00128u

Red Channel Displacement: x=-37, y=-52
Green Channel Displacement: x=-25, y=-35
Blue Channel Displacement: x=0, y=0

00137v

Red Channel Displacement: x=-8, y=-11
Green Channel Displacement: x=-6, y=-6
Blue Channel Displacement: x=0, y=0

00737u

Red Channel Displacement: x=-14, y=-49
Green Channel Displacement: x=-6, y=-15
Blue Channel Displacement: x=0, y=0

00757v

Red Channel Displacement: x=-5, y=-5
Green Channel Displacement: x=-3, y=-2
Blue Channel Displacement: x=0, y=0

00822u

Red Channel Displacement: x=-34, y=-125
Green Channel Displacement: x=-25, y=-57
Blue Channel Displacement: x=0, y=0

00888v

Red Channel Displacement: x=-1, y=-12
Green Channel Displacement: x=-1, y=-6
Blue Channel Displacement: x=0, y=0

00889v

Red Channel Displacement: x=-3, y=-4
Green Channel Displacement: x=-2, y=-1
Blue Channel Displacement: x=0, y=0

00892u

Red Channel Displacement: x=-5, y=-43
Green Channel Displacement: x=-3, y=-16
Blue Channel Displacement: x=0, y=0

00907v

Red Channel Displacement: x=0, y=-6
Green Channel Displacement: x=0, y=-2
Blue Channel Displacement: x=0, y=0

00911v

Red Channel Displacement: x=0, y=-13
Green Channel Displacement: x=0, y=-1
Blue Channel Displacement: x=-1, y=0

00992u

Red Channel Displacement: x=-19, y=-113
Green Channel Displacement: x=-13, y=-48
Blue Channel Displacement: x=0, y=0

01031v

Red Channel Displacement: x=-2, y=-4
Green Channel Displacement: x=-1, y=-1
Blue Channel Displacement: x=0, y=0

01043u

Red Channel Displacement: x=-19, y=-26
Green Channel Displacement: x=-10, y=0
Blue Channel Displacement: x=0, y=-15

01085u

Red Channel Displacement: x=-59, y=-111
Green Channel Displacement: x=-32, y=-42
Blue Channel Displacement: x=0, y=0

01734u

Red Channel Displacement: x=-50, y=-100
Green Channel Displacement: x=-29, y=-47
Blue Channel Displacement: x=0, y=0

01880v

Red Channel Displacement: x=-4, y=-14
Green Channel Displacement: x=-2, y=-6
Blue Channel Displacement: x=0, y=0

Results On Some Pictures I Liked

00295v

Red Channel Displacement: x=-1, y=-12
Green Channel Displacement: x=0, y=-3
Blue Channel Displacement: x=0, y=0

00300v

Red Channel Displacement: x=-1, y=-13
Green Channel Displacement: x=-1, y=-4
Blue Channel Displacement: x=0, y=0

00895v

Red Channel Displacement: x=-4, y=-16
Green Channel Displacement: x=-2, y=-7
Blue Channel Displacement: x=0, y=0