Image-Based Modeling and Rendering.
A central problem in computer graphics is producing images that appear photographic, thereby fooling people into believing they are viewing a real scene. While rendering techniques have advanced dramatically in recent years, we are still far from this goal of photorealism, largely because of the difficulty of constructing realistic 3D models. We propose to solve this problem by "importing" real-world objects and scenes from photographs and paintings. Towards this end, we are developing two classes of techniques, based on image morphing and 3D reconstruction, respectively. The first approach rearranges pixels in a set of input images in order to produce images of the scene from different camera viewpoints. This view morphing approach enables effects such as rotating a person's head in 3D from one photograph. We are also investigating voxel-based 3D reconstruction techniques to solve larger-scale visualization problems, such as producing building walkthroughs and flybys of complex landscapes by processing images from video camcorders.
3D Augmented Video Editing.
Imagine trying to remove a moving figure from one video sequence and paste it into another. While simple to describe, this operation is extremely difficult to achieve because of the need to manually trace moving contours in every frame of an image sequence. The goal of this project is to develop video editing tools that blend user-interaction with computer vision techniques in order to manipulate entire video sequences by touching only a small number of frames. In addition to cut-and-paste operations, we are interested in 2D Photoshop-style editing operations that propagate (e.g., applying lipstick to a moving face by painting it into a single frame) as well as 3D effects like changing camera viewpoint and illumination conditions.