15-463: Computational Photography

Project 2: Seam Carving

Overview

My implementation of seam carving follows closely to the paper, however my energy function differs slightly. Rather than using the Laplacian directly to get the gradient change, I used the Laplacian of the Gaussian. The Gaussian helps reduce the effects of noise and compression artifacts which were present in all of my source images since they were JPEGs.

I use rgb2gray to convert the image to grayscale before calculating energies. However the actual seam carving is done on the colored image.

A single seam carve takes a short amount of time, however hundreds done to each image does a couple minutes. This could be reduced with some smart use of the energy table, but I didn't have time to implement that bell and whistle. My seam carving function only carves one vertical seam, so to handle horizontal seams, I transpose the image in seamresize before passing it to seamcarve.

A bit of matlab trouble I ran into was when I was working with doubles and adding them and shifting them around and such, MATLAB would not display the image. This was because I did not convert it to uint8

Results

Original Cropped Notes
house_original house_carved

This is one of the example images given by the assignment. I did it to verify that my code worked. The trees did not suffer as much from the vertical carving as the reference carved image did. I believe this is due to my slightly smarter energy function.

The horizontal carving in this photo shows a great advantage of this technique. In this image, the smoke is preserved but it has flattened out a bit, as would be expected from horizontal carving.

There might also be slight aliasing effects going on.

To my surprise, much of the snow did not get carved. I believe this is due to the small blades of brown grass poking out of the white snow causing large color gradients.

Picture credit to Jim McCann

umbrella_original umbrella_carved_1

This is a picture of an umbrella frame set against the sky.

This is a simple test of the carving algorithm where I pushed it to crop out the extraneous sky. However, as we will see later, this will contrast quite strongly against the results of asking the algorithm to crop far too much information.

Take a look at the bottom-right corner. There is some artifact there. We will see this play a big role in the later image.

Picture credit to my good friend Jason Lee

umbrella_original umbrella_carved_2

Here we ask the algorithm to work past its limits. 300x300px is not enough to contain the entire umbrella frame so the seam will inevitably destroy the look of the umbrella. The previously straight umbrella has been bent out of shape. It almost looks like it's been degraded by heat and the metal is starting to melt. Neat effect.

Look at the bottom-right artifact. I believe the high change in gradient there caused that seams containing that area to never be cut so it was preserved and managed to push that bit of the umbrella frame too far to the left

Picture credit to my good friend Jason Lee

reflection_original reflection_carved

This is a picture of a fountain on the campus of the University of Southern California.

I wanted to see what the algorithm would do to reflections. Turns out, it didn't do too badly. As with the example sidewalk photo, it seems like the angle of the camera when the photograph was taken has changed. Additionally, we can ascribe the wavy lines that really were artifacts of seam carving to the water having its way with light waves. ;)

Picture credit to my good friend Jason Lee

birds_original birds_carved

I chose this picture because I wanted to see what the algorithm would do to photographs where the subject was sort of camouflaged with a busy background (ie, high contrast background). I chose a window that would adequately contain the entirety of the birds.

The results make it look like the birds woke up too late and couldn't catch any worms. The house's windows got really messed up. The branches still look lifelike because trees can be weird like that.

We can see that the seams traveled down the bird. This is likely because the bird is mostly a single gradient of color whereas the other paths included flowers, houses, and trees. We can also see that a lot of the open space between the leaves went away, likely due to the single color of the house exterior.

Picture credit to my good friend Jason Lee

hockey1_original hockey1_carved

I chose this picture because I love hockey and I love to laugh at Ovechkin (of the Capitals, our conference rivals) in his disastrous Olympic play for the Russian team. LETS GO PENS!

I wanted to make the scene more dramatic by pushing all the players together by removing that small patch of ice in the center. That's about 67 pixels wide, so I decided to remove 160 columns.

Aside from the straight lines painted in the ice, it worked out pretty well here. All the players are closer together and even the puck is closer. The Olympic rings are slightly distorted, but still retain their circular shape. However they are no longer the same size. There also seems to be less ice behind the net. I'm not sure I like that (as a hockey fan).

So close Ovie, so close. And with my picture, you're even closer!

Picture credit to Kip Photography

waterway_original waterway_carved

I chose this picture because I wanted to change the perspective lines generated by the waterway shores. To do that, I decided be EXTREME and remove all but 250 columns.

The waterway is much narrower now. Because of the original unevenness, any wavy lines caused by the seam carving simply look natural.

Picture credit to my good friend Jason Lee

broadwaytower_original broadwaytower_carved

I chose this picture because I wanted to crop out the center bit between the man and the castle. This kind of effect cannot be achieved with traditional border cropping and would require some Photoshop work for somebody cropping out the center by hand, especially to preserve the clouds.

Picture credit to Newton2

ellenpage_original ellenpage_carved

I chose this picture because Ellen Page is beautiful.

Oh my god! Arghhh! Now I regret doing this assignment.

Anyways, my guess is that the background lights had too high a gradient, so the seam went straight for her face.

Picture credit to Josh Jenson

shuttle_original shuttle_carved

I chose this picture because I love the shuttle and there is a large section on the right that is croppable. Let's see how the algorithm does.

The algorithm did really well. The beam of light is barely bent, as would be expected since its suddenly very close to the shuttle

I'm not entirely sure of the source of this image because I downloaded it for a wallpaper a while ago. If I had to bet, it would be NASA's APOD.

Valid XHTML 1.0 Strict