15463-Computational Photography

Picasso Guernica

Project 2: Content Aware Resizing of Images using Seam Carving

Varun Ramakrishna

guernica2



Introduction

In this project the techniques for resizing images in a content-aware fashion were implemented. The method used was the creative yet simple Seam-Carving technique by Shai Avidan and Ariel Shamir in Seam Carving for Content-Aware Image Resizing, SIGGRAPH 07. The basis of the technique is to find paths of lowest importance as defined by an energy function and remove or add them depending on whether we want to reduce or enlarge the image respectively, using dynamic programming to search for the path or 'seam' as shown below applied to an image of Picasso's Guernica.

guernicaseam1 seam2 seam3
 
The Energy function used in the project was the magnitude of the value of the gradient at each pixel in the image. The following methods were implemented:

Vertical Seam Carving
Horizontal Seam Carving
Object Deletion & Seam Insertion

Vertical Seam Carving

The test images were resized using the seam carving algorithm by removing vertical seams. Here are some results:
 
            Original                                                                                                                        Vertical Seam carved
bird1  birdr1

                            Original                                                                                                                Vertical Seam carved
lamayuru lama2

                       Original                                                                                                          Vertical Seam carved
giza giz2

                   Original                                                                                                              Vertical Seam carved
beach2 beachvsc

                Original                                                                                                                Vertical Seam Carved
flower flowervsc

Horizontal  Seam Carving

For horizontal seam carving, the Image was transposed and passed to the vertical seam carving function and then transposed again to get the result


                                                                         Original
birds
                                                                         Horizontal seam carving
birdsr


                                                                                     Original
lighthouse
                                                                                      Horizontal Seam carving
lighthouser


                                                                         Original
moon
                                                                         Horizontal Seam carving
moonr


                                                                                        Original
night
                                                                                         Horizontal seam carving
nightr




Object Deletion & Seam Insertion

Object deletion was implemented by providing the user with an option to select an area in the image to delete. The selected area is given negative energy values thereby attracting minimum energy seams towards it. After the selected area is deleted by seam removal, the image is enlarged y seam insertion. Images were enlarged by finding the seams of minimum energy and duplicating them. To prevent the same seam from being duplicated over and over again, we find the first k seams for removal and duplicate them to enlarge the image. A few examples are shown below:

                   Original                                                                Deletion area selected                                     Object deleted
beach2 beach2delpoints beach2delgood


                                                                         Original
london
                                                                         Deletion area selected
londonp
                                                                         Object deleted
londondel

                         
                                                                         Original
station
                                                                         Object Selected
stapoints
                                                                          Object deleted
statdel



Some examples of failure:

The seam removal procedure worked well for some images, mainly in images with wide uniform areas, or areas with a lot of redundancy. It produced noticable artifacts in images where the content is distributed.
   

         jeep jeep1
                                                                              
Artifacts are noticeable in the image on the right, the truck has been reduced awkwardly and the man in the left of the image has also been awkwardly reduced in size. Another example is shown below where the seam removal has distorted the shape of the wall in the background of the image.
stones  stones1

Just for Fun : The strokes they didn't have to paint

I performed seam removal on images of some of the most famous paintings in the world such as Guernica by Pablo Picasso ( Header image) and a few others. With my apologies to the great masters, here is a re-interpretation of some great works with the help of seam carving:

Rene Magritte's Golconde


golconda
                                                                                                                         Golconda1


Picasso's Guernica
(
I like the way all the subjects have been bunched together in the seam carved version, and you don't know what are artifacts and what is cubism :) )

gu                                                                                 gu2

Salvador Dali's Persistence of Time

da1s                                                       da2


Image sources         

The images used were from three sources. Creative commons for the art pictures, pictures taken by me of the Leh-Ladakh district in India and flickr (Joanot, Marlon Mayer, jdeboer152, neloqua)