# 15-463 Computational Photography "Images of the Russian Empire" - Sept. 12, 2007

## Anna Michalska

### Colorizing the Prokudin-Gorskii Photo Collection

Low Resolution - Exhaustive Search - Sum of Squared Differences

 Image: '00228v.jpg' Image: '00011v.jpg' Displacement: Red: x = 5 y = 8 Green: x = 3 y = 1 Displacement: Red: x = 4 y = 8 Green: x = 2 y = 2

 Image: '00385v.jpg' Image: '00095v.jpg' Displacement: Red: x = 0 y = 10 Green: x = 0 y = 4 Displacement: Red: x = 0 y = 14 Green: x = 0 y = 3

 Image: '00234v.jpg' Image: '00091v.jpg' Displacement: Red: x = 1 y = 14 Green: x = 0 y = 4 Displacement: Red: x = 3 y = 15 Green: x = 2 y = 7

 Image: '00206v.jpg' Image: '00215v.jpg' Displacement: Red: x = 1 y = 1 Green: x = 1 y = 6 Displacement: Red: x = -3 y = 9 Green: x = -2 y = 4

 Image: '00032v.jpg' Image: '00086v.jpg' Displacement: Red: x = 1 y = 9 Green: x = 1 y = 4 Displacement: Red: x = 5 y = 13 Green: x = 3 y = 6

 Image: '00498v.jpg'' Image: '00646v.jpg' Displacement: Red: x = 3 y = 10 Green: x = 3 y = 4 Displacement: Red: x = 4 y = 14 Green: x = 2 y = 6
 Image: '00165v.jpg' Image: '00704v.jpg' Displacement: Red: x = 1 y = 9 Green: x = 1 y = 0 Displacement: Red: x = 3 y = 13 Green: x = 2 y = 6

 Image: '00858v.jpg' Image: '00083v.jpg' Displacement: Red: x = 4 y = 13 Green: x = 3 y = 6 Displacement: Red: x = 1 y = 6 Green: x = 2 y = 2

 Image: '01728v.jpg' Image: '01039v.jpg' Displacement: Red: x = 1 y = 18 Green: x = 1 y = 8 Displacement: Red: x = 2 y = 11 Green: x = 1 y = 5

 Image: '00084v.jpg' Image: '31421v.jpg' Displacement: Red: x = -9 y = 11 Green: x = -4 y = 4 Displacement: Red: x = 0 y = 13 Green: x = 0 y = 8

 Image: '00017v.jpg' Image: '00056v,jpg' Displacement: Red: x = -3 y = 1 Green: x = -2 y = 1 Displacement: Red: x = 1 y = 13 Green: x = 1 y = 5

 Image: '00362v.jpg' Image: '00872v.jpg' Displacement: Red: x = -5 y = 11 Green: x = 0 y = 5 Displacement: Red: x = -2 y = -2 Green: x = -1 y = -3
Called function: [Goffset Roffset] = color('image_name', window_size)
image_name - the name of the image in the Photos/ directory
window_size - the size of the window of possible displacements
Goffset - the alignment vector of the green channel
Roffset - the alignment vector of the red channel

The first value of the alignment vector is the y (vertical) component
and the second value is the x (horizonal) component.
Values of y component increase from top to bottom of the image
and values of x component increase from left to right of the image.

With the exception of 01728v.jpg where the window_size is set to 20,
in all of the remaining ones we do exhaustive search on [-15x15] area.

To remove the white/black edges that cause error in the alignment process,
after separating the RGB channels, we crop 1/15 of all images from four sides.
For images: 31421v.jpg and 00084v.jpg we only have to crop 1/25 from their top.

High Resolution - Image Pyramid - Sum of Squared Differences

 Image: '01044u.tiff' Displacement: Red: x = 0 y = 22 Green: x = 1 y = -11
 Image: '01845.tiff' Displacement: Red: x = 34 y = 84 Green: x = 23 y = 37
 Image: '00797u.tiff' Displacement: Red: x = 14 y = 144 Green: x =12 y = 67
 Image: '01443u.tiff' Displacement: Red: x = 44 y = 79 Green: x = 21 y = 32
 Image: '00794u.tiff' Displacement: Red: x = 17 y = 123 Green: x = 16 y = 53
 Image: '00033u.tiff' Displacement: Red: x = 14 y = 104 Green: x = 10 y = 52
Called function: [Goffset Roffset] = color('image_name', window_size)
image_name - the name of the image in the Photos/ directory
window_size - the size of the window of possible displacements
Goffset - the alignment vector of the green channel
Roffset - the alignment vector of the red channel

The first value of the alignment vector is the y (vertical) component
and the second value is the x (horizonal) component.
Values of y component increase from top to bottom of the image
and values of x component increase from left to right of the image.

### Difficulties

 Image: '00153u.jpg' Displacement: Red: x = -810 y = 196 Green: x = 24 y = 69
In this case, the algorithm implementing Gaussian pyramid returned

 Image: '00153u.jpg' Displacement: Red: x = 16 y = 72 Blue: x = -24 y = -69
In order to improve the alignment, a new function colorrev() was written.
Instead of aligning a red and green filter to the blue one (as in color())
it aligns red and blue filter to the green one.

### Write-up

Aim:
Using image processing techniques automatically colorize images of the Russian Empire
from the Prokudin-Gorskii photo collection (both high and low resolution versions).

Process of completion:
- Extract three color channel images from one glass plate
- Place them on top of each other in the right order: red, greeen, blue
- Remove all white and black edges around the image
- Find the best alignment between the filters using
either Sum of Squared Differences (SSD) method of measurement
or Normalized Cross-Correlation (NCC) method of measurement
- For large resolution images implement the image pyramid
- Crop final image to the area shared by all three layers

### Bells & Whistles

Image Cropping

 Image: '00646v.jpg' Image: '00646v.jpg'
In order to properly crop the picture, two points are found.
The first one - the lowest value in the upper left corner
and the second one - the highest value in the lower right corner.
These two numbers form the diagonal of the cropped image.

 Image: '00362v.jpg' Image: '00362v.jpg'
Cropping to the areas shared by all three layers

 Image: '00084v.jpg' Image: '00084v.jpg'
Cropping to the areas shared by all three layers

 Image: '00086v.jpg' Image: '00086v.jpg'
Cropping to the areas shared by all three layers

 Image: '00498v.jpg' Image: '00498v.jpg'