15-862 Computational Photography

Project 5: Single-View Reconstruction
(Tour Into the Picture)
by: Lisa Chan

Project Details:

For this project, I followed the equations and methods given in Siu-Hang Chu's thesis, even though the method was derived from the article by Horry et al. For selecting the back wall and the vanishing point, I used the GUI provided by professor Efros. The main challenge in this project was in warping the walls of the images into the 3D box. Here are the procedures I used to create complete this project:
  1. Have user input coordinates for the backwall and vanishing point.
  2. Extract the 5 polygons that comprise of the walls, ceiling, and floor.
  3. Use a mask to extract each polygon from the original image.
  4. Warp each polygon back into a rectangle.
  5. Crop off any major black borders around the warped images.
  6. Calculate the depth and height of the 3D box.
  7. Define the geometry of the 3D box and the surfaces of the walls in 3D.
  8. Warp the rectangular images onto the walls of the 3D box.
In extracting the foreground objects in the image, I prepared the image in Photoshop to create a mask for the object. Unfortunaltely, I couldn't get Matlab to work with the alpha channel. Matlab would either just crash or just refuses to make parts of the foreground image transparent. Nevertheless, I've included my results with a box of the foreground object.
In the fly-through movie that I've created, I didn't use Matlab to output the avi file. I simply used camdolly to have the camera move, then output each frame of the movie with frame2im, and finally reconstructed the movie using Adobe Premiere.


1. Standard Picture Provided in Class:

For this picture, I used a focal length of 500, which gave pretty good results.

Input Picture
Input Points

Basic Novel Views

When I extracted the dog or lion from the image, I simply traced his outline in Photoshop to create the mask. Next I used the clone tool in Photoshop to fill in the floor to cover up where the dog / lion has been.
Original Picture
Mask
Background Picture

Foreground Extraction Views

For the fly-through movie, it took me a lot of time playing with camdolly so that it would fly in to the appropriate corners and have a smooth transition.
Fly-Through Movie


2. My Chosen Pictures:

This is one of the pictures in Professor Efros's Flickr image dataset. The image worked really well except for the dome ceiling. The semi-circular glass panel became very distorted. However, the corners at the back of the room looks really good. Here, I used a focal length of 400.

Input Picture
Input Points

Basic Novel Views

For this fly-through movie, I modifed the camdolly parameters from the previous movie to make the movement suit this image.
Fly-Through Movie


I took this picture on an early Sunday morning so that nobody would be around at NSH. The focal length I chose for this is 600. I think it looks better when I use 800, but then it would be hard to capture novel views since the box would be so long.

Input Picture
Input Points

Basic Novel Views


I obtained this image from Alan L Franklin Art Gallery online. Since this picture is very similar to the picture given in class, it worked really well. The focal length I used was 200 for this image.

Input Picture
Input Points

Basic Novel Views