SEAM CARVING
Computational Photography, Proj #2
By Emine Begum Gulsoy
This project deals with optimizing the image resizing,
according to the importance of each pixel within the image. To define the
significance of each pixel, an energy function is calculated. This energy
function, then, is used as a starting point in finding a path of pixels through
the image which will give the minimum sum; in other words defining a path of
relatively insignificant pixels. The only requirement for the basic part of
this project is to choose exactly one pixel from each row at a time.
The program I have written first, takes a colored image and
breaks it into its RGB channels. Then using the information the image is
converted into a grayscale image. General matrices for getting directional
derivatives are defined and convolved with the grayscale image. The results of
X and Y directions are summed to come up with the basic energy function.
Next part involves calculating the minimum sum and the
possible paths to achieve them. To do this first two rows of the energy
function are extracted and their possible summation combination is calculated
by summing over shifting one pixel in +x and -x directions. A minimum of the
three sums is taken to be the minimum sum array. Then the same process is
repeated using the minimum sum array and third row, and so onÉ Meanwhile how to
go from pixel one in row one to the neighboring pixel, which will give the
minimum sum, is recorded in a path array. Once the image is scanned from top
the bottom and the final minimum sum is known a backward process is applied to
eliminate the appropriate pixels.
Finally, the image size is shortened by one pixel and the
same process is repeated according to how much one wants to resize the original
image.
The code is originally written for vertical seam carving,
however by activating some of the commented-out parts of the code and giving
the transpose of an image as an input, it is possible to get horizontal carving
as well.
This program has been written in IDL- Interactive Data
Language, you will need either IDL or OpenGL to process it.
Here are the results for basic vertical and horizontal seam
carving:
Original Image[1]
Left: Seam carved image, reduced in x size by 50
pixels
Right: Original picture resized using Adobe
Photoshop, reduced in x size by 50 pixels
Left: Seam craved image, reduced in x size by 100
pixels
Right: Original picture resized using Adobe
Photoshop, reduced in x size by 100 pixels
Comparing the results with the normal resizing techniques,
seam carving- at least for this picture- does a nicer job in maintaining the
basic element of the image- the pink flower. Adobe Photoshop squishes the
flower a little whereas seam carving squishes the background leaves and leaves
the flower more robust. This effect gets even better with increasing reducing
sizes. Comparatively the second set of pictures have more significant
differences than the first set.
Original Image[2]
Image seam carved by 50 pixels in x direction
Left: Image resized by 50 pixels in x direction
using the CONGRID built-in function of IDL
Right: Image resized by 50 pixels in x direction
using Adobe Photoshop
It is not very visible but when one looks at these images by
overlapping them, it turns out that the seam carved one preserves the boat a
little better than both the congrided and the photoshoped images. It does a
relatively worse job with the background where the towers are a little eaten
away. However, things change when the extent of resizing is increased:
Image seam carved by 200 pixels in the x direction
Left: Resizing by CONGRID built-in function, 200
pixels in the x direction
Right: Resizing by Photoshop, 200 pixels in the x
direction
Both Congrid and Photoshop are doing a better job in
preserving especially the background- the towers- of the image. The ships are
almost equally distorted. This is a picture on which this energy function works
better with small readjustments in size.
Original Picture[3]
Image seam carved by 100 pixels in the x direction
Original Image[4]
Image seam carved by 50 pixels in y direction
Original Image
Image seam carved by 50 pixels in the x direction
Original Image
Image seam carved by 100 pixels in the x direction
Original Image
Image seam carved by 100 pixels in the x direction
Original Image[5]
Image seam carved by 100 pixels in the y direction
Original Image
Image seam carved by 100 pixels in y the direction
Image seam carved by 250 pixels in the y direction
So far the example pictures have been working nicely- more
or less. Not all the images I have tried using gave such nice results. The
above images are rather smooth in the sense that they have almost uniform
things that span a wide region- continuous clear sky, seaÉ etc. Images with
more details tend to give worse results with the simple energy function I am
using. Here are some examples where my code failed to work properly.
Original Image
Image seam carved by 50 pixels in the y direction
Image seam carved by 167 pixels in the y direction
(half size)
In the above images we can observe the artifacts that appear
as the image gets smaller. Whereas the image reduced only by 50 pixels looks
ok, the image reduced to half its size contains too many artifacts.
Original Image[6]
Image seam carved by 50 pixels in x direction
Image seam carved by 100 pixels in the x direction
Left: Image resized by 100 pixels in the x
direction by CONGRID function
Right: Image resized by 100 pixels in the x
direction by Adobe Photoshop
The cat in the front doesnÕt get affected when the image is
seam carved, however the black cat thins down and is distorted. When one goes
to higher resizing levels, the shrinkage in black cat is even worse. However,
when I tried resizing with other methods I actually saw that at least the front
cat is preserved and not squished with seam carving and Congrid, but looks
really weird when resized by Photoshop.
Original Image
Left: Image seam carved y 100 pixels in the x
direction
Right: Image resized by 100 pixels in the x
direction using CONGRID
As can be seen, although seam carving does a good job in
eliminating 100 pixels very easily, it is not strong in preserving straight
lines in the image. The main problem with the lines in this picture is that
they are in the front and not in the background such that they make up a very
important element of the image. Pixel wise they may not be important but this
does not coincide with the way we perceive the image. Below is another example
about the same problem, even more extreme.
Original Image
Left: Image seam carved by 100 pixels in the x
direction
Right: Image resized by 100 pixels in the x
direction using Adobe Photoshop
Bells and Whistles
Program
vs Human
The image
was reduced by 100 pixels in the x direction using Adobe Photoshop. Main parts of
the image has been identified as:
-
The
boat
-
The
buildings (mainly the palace) at the background including the minarets and the
towers
-
The
walkway that is slightly visible in between the green region in the middle of
the image
First, the
boat was shifted to left, right under the palace, keeping its size. Next the
walkway was shifted to the left, being connected to the palace wall. Then the
shifted elements were erased from the original image. Finally slight color
adjustments were made to blend each piece into one another. The best thing
about hand carving has been the fact that the boat, the tower and the minarets
were much better preserved. Here is the result:
Original Image
Left: Hand seamed image using Adobe Photoshop
Right: Seam carved image using the program IÕve
written
Seam Insertation
Original Image
Image stretched by 100 pixels in x direction
Original Image
image stretched 100 pixels in y direction
Original Image
Image stretched 100 pixels in the x direction
Image stretched by 100 pixels in the y direction
Original Image
Image stretched by 100 pixels in x direction
Image stretched by 100 pixels in y direction
I have found the indexed images over the web. Most of them
were from flickr.com but some are from Google image search. Unfortunately I
couldnÕt relocate the images and therefore cannot acknowledge the photographers
except for:
[6] Camil
Tuncan,
http://www.flickr.com/photos/camil_t/1427836797/in/set-72157602116217702/
Rest of the images are mine :)