Assignment: Seam Carving

Ronit Slyper


I implemented the basic idea of "Seam Carving for Content-Aware Image Resizing" by Shai Avidan and Ariel Shamir.
The code to do this consisted of three files:

Energy Function

I began with a gradient computed using nlfilter; this tack was way too slow.
I tried a laplacian, gaussian, and unsharp filters on both the images and energy, but none proved better that the gradient: the gaussian removed high frequencies, so the result wasn't useful; the laplacian and unsharp emphasized edges too much.
I also tried computing the gradient, then eroding the result to widen the ``don't go here'' borders between objects, in hopes that this would prevent the failure case where seams would break the structure of objects. Artifacts of the strel shape were too great, however, and I did not want to blur the gradient.
In a similar vein, I used a canny edge detector on the image, imdilated the resulting edges, and added it to the gradient energy-matrix. The hope was this would prevent edge crossings by making edges unappealing, and thus prevent distortion of structures. Results were

Why so bad? The cause is an imperfection in the scanned image that results in many of the seams heading for the middle, which has a zero gradient patch:

The next image worked better:

It had no such imperfections, so the edge detection helped more. Using a simple gradient energy function:
and a gradient with enhanced edges:

The suits of the men show less distortion.

In all following images I used a gradient computed via sums of convolutions.


I learned matlab's gui-creation toolkit (GUIDE); it is similar to Visual Basic and thus easy to get a grasp of. Getting the axes object to resize with the image was a long and mentally anguishing trip. Here is the GUI in action:

Viewing the energy functionViewing the imageViewing a larger image.Object selection for removal

Bells & Whistles

Make it fast; Compose an image and compare to seam carving; Remove objects (using seam carving, without seam insertion); Matlab GUI for viewing and object removal.


I resized images in both directions, sequentially. (However, if an image was an obvious failure in the first direction, I often didn't take the time to continue resizing in the other direction). All images are shown at actual size (no standard scaling of results!), with the original on the left and resized version on the right.


Maurice Denis, MoMA.When important objects are smooth, they end up looking rather emaciated.
flickr + matlab fun. This picture is both a success and a failure. The seam carving is flawless; however, the gradient fails to reflect a human notion of importance, and thus large stretches of high-frequency beach are preserved.
flickrfun. The duck is very smooth compared to the high-frequency water, and thus ends up doing a strange dance move.
MC Escher. Seams pass through lines, removing structure.
Felix del Marle, MoMA. This image is both a success and a failure. It has the same problem as the previous Escher work. However, new art is created!.
George Grosz, MoMA. I was surprised this came out so bad, with the same problem as above. The smooth regions are still noisy, colorwise, so seams pass through edges.
flickr. High frequency background means the lady's head is seam carved.
Leopold Survate, MoMA. Too noisy.
Lyubov Popova, MoMA. So much for being clever.


Kupka, MoMA. Great picture for a gradient energy function.
MC Escher. Self similarity! A great place for scene carving. The results look suspiciously like scaling, though.
MC Escher. Large monochrome blocks are easy to carve.
MC Escher. This distorts the structure somewhat, but you don't notice it at first, since Escher paintings are disorienting anyway.
Matisse, MoMA. Weight loss via seam carving.
Van Gogh, MoMA. Landscaping via seam carving.

Object Removal

It is difficult to find images where object removal will work well.
MC Escher. I roughly painted out one of the fish, resized the image, then refined it by painting out the tail in the bottom, and resizing again.
self. I roughly delineated the man blocking my view, and seam-carved him out in both directions.
self. At left, I perform the same task in the Gimp, using the clone brush. At right, I duplicate one of the cuter occupants to replace the removed guy. I found that attention to the shadows results in an astounding leap in the images' credibility.