15-463 Computational Photography
"Seam Carving" - Sept. 29, 2007

Anna Michalska

Content-Aware Image Resizing

Horizontal Resizing
Image: 500 x 375
Source: Flickr Photo Download
Image: 200 x 375

Image: 500 x 375
Source: Flickr Photo Download
Image: 200 x 375

Image: 500 x 375
Source: Flickr Photo Download
Image: 200 x 375

Image: 500 x 375
Source: Flickr Photo Download
Image: 200 x 375

Image: 500 x 375
Source: Flickr Photo Download
Image: 200 x 375

Image: 500 x 375
Source: Flickr Photo Download
Image: 250 x 375

Image: 500 x 375
Source: Me
Image: 300 x 375

Image: 500 x 375
Source: Flickr Photo Download
Image: 300 x 375

Called function: image_resize('image_name', direction, seams, 'image_resized')
image_name - the name of the image in the Photos/ directory
direction - flag 'h' horizontal image resizing, column removal
seams - the number of columns to be removed from the image
image_resized - the name of the resized image in the Photos_Results/ directory

Vertical Resizing
Image: 500 x 375
Source: Me
Image: 500 x 225

Image: 500 x 375
Source: Me
Image: 500 x 200

Image: 500 x 375
Source: Flickr Photo Download
Image: 500 x 200

Image: 500 x 375
Source: Flickr Photo Download
Image: 500 x 175
Called function: image_resize('image_name', direction, seams, 'image_resized')
image_name - the name of the image in the Photos/ directory
direction - flag 'v' - vertical image resizing, rows removal
seams - the number of rows to be removed from the image
image_resized - the name of the resized image in the Photos_Results/ directory

Write-up

Aim:
Using image processing techniques effectively resize an image
(either horizontally or vertically) considering the image content.
(Based on paper entitled "Seam Carving for Content-Aware Image Resizing"
written by Shai Avidan and Ariel Shamir - SIGGRAPH 2007)

Process of completion:
- Horizontal Resizing
- Read in an image that needs to be resized horizontally
- Determine the importance of every pixel by means of an energy function
- Find the horizontal seam of lowest importance (dynamic programming)
- Remove the seam by shifting some pixels one position to the left in every row
- Decrease by 1 the number of the remaining columns to be removed
- Repeat the same procedure until the image shrinks to desired width
- Store the resized image

- Vertical Resizing
- Read in an image that needs to be resized vertically
- Rotate the input image by 90 degrees counterclockwise
- Follow the same procedures as for Horizontal Resizing
- Rotate the resulting image by 90 degrees clockwise
- Store the resized image

A similar approach is taken for effective Horizontal and Vertical Image Enlarging.
All seams of lowest importance are duplicated by averaging the colors of their
west and east neighbors (horizontal enlarging) or north and south neighbors (vertical enlarging).

Difficulties

Horizontal Resizing
Image: 500 x 375
Source: Me
Image: 200 x 375
After resizing the above image, the majority of pixels representing trunks of both palms are removed.
These trees are important components of the image (they contrast with the background)
and, therefore, by removing some parts of them, an algorithm returns an undesired result.

Image: 500 x 375
Source: Me
Image: 250 x 375
After removing 250 vertical seams (exactly half of the image) the most important elements of the picture
(the building on pier, sailing boat, mountains, bridge) are not erased but correctly preserved.
Only the shape of the mountain on the left is distorted - it does not look very real.

Vertical Resizing
Image: 500 x 375
Source: Me
Image: 500 x 200
Clouds on the blue sky form a simple background that is not distored by the vertical resizing.
The horizontal surface of the water between palms forms a more complex structure - its geometry is not preserved.

Image: 500 x 375
Source: Flickr Photo Download
Image: 500 x 175
The white diagonal line at the bottom of the original image contrasts with the dark background.
After resizing the picture, this line is no longer straight but more 'blocky.

Bells & Whistles

Animated Finding of Horizontal Seams of Lowest Importance

Called function: color_seam('image_name', direction, seams, 'image_resized')
image_name - the name of the image in the Photos/ directory
direction - flag 'v' - vertical image resizing, 'h' - horizontal image resizing
seams - the number of rows/columns to be removed from the image
image_resized - the name of the resized image in the Photos_Results/ directory

Animated Vertical Resizing


Horizontal Image Enlarging
Image: 500 x 333
Source: Flickr Photo Download
Image: 675 x 333

Image: 500 x 333
Source: Flickr Photo Download
Image: 675 x 333
Called function: insert_seam('image_name', direction, seams, 'image_resized')
image_name - the name of the image in the Photos/ directory
direction - flag 'h' - horizontal image resizing, columns insertion
seams - the number of columns to be inserted to the image
image_resized - the name of the resized image in the Photos_Results/ directory

Vertical Image Enlarging
Image: 500 x 333
Source: Flickr Photo Download
Image: 500 x 433
Called function: insert_seam('image_name', direction, seams, 'image_resized')
image_name - the name of the image in the Photos/ directory
direction - flag 'v' - vertical image resizing, rows insertion
seams - the number of rows to be inserted to the image
image_resized - the name of the resized image in the Photos_Results/ directory