Heather Strong
15-463 Fall 2008
Project 5: Tour into the Picture: Single View Modeling

Given an image of an area with a ceiling, 2 side walls, a floor, and a backwall (or, an image with a similar shape but which may be missing the ceiling or walls) one can define rough geometry for the scene, then texture map the original image onto the geometry, as described in Tour into the Picture by Horry et al. The geometry is reconstructed by finding the locations of the corners of the room in 3D space using similar triangles. The planes they form are then textured using pieces of the original image, which have been transformed into a rectangular shape via a homography. Then, one is able to move through the scene and capture pictures that appear to be taken from a different position. Because the geometry is only a very rough approximation of the original scene and the textures may have had to be stretched significantly, the scene will, of course, not be perfect. But it is good enough to create some very interesting and effective effects.

I wrote this program in Matlab, as was suggested. When I rendered my scenes, strange artifacts appeared, such as patches of brightly colored pixels. These did not exist on the original photographs I had taken. In addition, the images, as they appear in the 3D scene, also seem to be of a much lower resolution than the textures I actually rectified and placed on the walls. I can only assume this is some sort of optimization on the part of Matlab's renderer.

Original Photo
Original Photo
Original Photo

Bells and Whistles

Foreground Objects

Additional planes can be added to the rough geometry in order to have foreground objects in the image without distortion. In this case, an alpha channel needs to be created so that the foreground objects are properly masked out. The below left image shows an example of the foreground geometry as it begins to stand out against the increasingly distorted rectangles in the background. The below right image is a unique shot of the scene that has been warped enough so as to remove the distorted rectangles from the shot.

Original Photo