My initial implementation was fairly simple. I started by shifting the image by -15 to +15 pixels in both the X and Y direction, calculating the sum of squared differences between each channel, until I was confident that my algorithm worked. Even without any color adjustment or cropping, SSD yielded great results for the smaller images. When I moved on to the higher resolution images, however, I ran into a lot a difficulty. Without using an image pyramid, the code was far too slow to be usable; I implemented a recursive function which reduced the image until it was under a certain size, then aligned the channels as a coarse grain search. As I traversed upwards, I performed small searches at each level to refine the alignment. However, I kept getting terrible results, no matter what I tried. I eventually realized that my problems were coming from the borders of the images; with a smaller search window, more of the border was being used in the alignment, and this didn't always match up in the larger images. If I cropped the borders of the image, taking 10% off all four sides, the alignment improved dramatically. At this point, my code worked fine, but it was slower than I wanted. As a last step, I added a window boundary to my search. On every level, instead of attempting to match the entire picture, my algorithm only compared a 300 by 300 pixel window in the center, since I assumed that there was enough detail here in most of the images. This proved particularly useful in the highest levels of searching. Even with a small shift window, the images were so large that the code was too small without limiting the search size. I played with the parameters for a while and determined that 300 pixels provided the best tradeoff between performance and execution time.
|
|
Sample Images - JPG |
|
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
Sample Images - TIF |
|
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
Extra Images |
|
 |
|
 |
|
 |
|
Additional Tweaks |
|
As for additional tweaks, I worked on removing some of the borders and playing with the color parameters. I did a pass at the beginning of the algorithm to cut 10% off each side, removing the black and white edges, but ran into another issue after my alignment finished. I used circshift to move the red and green channels, which would move pixels from one edge to another on each channel. This created unnatural artifacts, so I cropped the image to remove those edges based on the biggest shift in each direction.
|
|
With a bad border:
|
|
With the shifted reigons cropped out:
|
|
I also tried to implement white balancing for more natural colors. I tried both white world and grey world balancing, with limited success. I settled on white world in the end; the effect wasn't particularly noticable, but it did slightly improve the appearance of the image. Grey world had a significantly more visible effect; it affected the images with much greater strength, but often yielded undesirable results. Two such results are shown below. In the first image, the red tint is removed from the sky, resulting in a much more natural color, but the ground has lost all of its contrast and is washed out. In the second image, the color becomes very unnatural. The image has very little red in it, so scaling to grey pulls up the red channel significantly, ruining the color. |
|
Grey world: more natural color, but the edge of the carpet is washed out:
|
|
Grey world: Red channel is pulled up too high
|
|
White world: Colors are a bit more natural
|
|
Offsets: File 1: 00029u.tif |
|
