3D Object Manipulation in a Single Photograph using Stock 3D Models

Natasha Kholgade 1Tomas Simon 1Alexei Efros 2Yaser Sheikh 1
1 Carnegie Mellon University2 University of California, Berkeley

News
Check out the news on our work on editing photographs in three dimensions:

TechCrunch


We were interviewed about our work by WESA (the NPR radio station in Pittsburgh), and we were featured live on the local radio station KQV 1410 AM.

Full Length Video

Abstract
Photo-editing software restricts the control of objects in a photograph to the 2D image plane. We present a method that enables users to perform the full range of 3D manipulations, including scaling, rotation, translation, and nonrigid deformations, to an object in a photograph. As 3D manipulations often reveal parts of the object that are hidden in the original photograph, our approach uses publicly available 3D models to guide the completion of the geometry and appearance of the revealed areas of the object. The completion process leverages the structure and symmetry in the stock 3D model to factor out the effects of illumination, and to complete the appearance of the object. We demonstrate our system by producing object manipulations that would be impossible in traditional 2D photo-editing programs, such as turning a car over, making a paper-crane flap its wings, or manipulating airplanes in a historical photograph to change its story.


Paper (14.4M) | Supplementary (35.7M) | BibTeX
Short Video (1280x720, 23.8M) | Full Length Video (1280x720, 150.9M) | Alignment Example
OS X (Mavericks) Executable Code and Examples | OS X Source Code (under the GPLv2 license)