3D Shape + Morphing

Man, Sofa and Ball

Input Sequence

Input sequece at one camera position among 49 cameras.

MPEG movie file (1,750KB) Complete video sequence at one camera position

3D Shape

Reconstructed 3D shape using Multi Baseline Stereo method.

MPEG movie file (835KB)

Virtualized Video Sequence

Output video sequence using 3D shape and morphing.

MPEG movie file (1,145KB) Complete virtualized video sequence at virtual view point

In our "3D Room", we can create a virtualized video sequence from captured images of 49 cameras as shown above. To create the virtualized video sequence, we use 3D shape, reconstructed from the video sequence using Multi Baseline Stereo method and morphing technique. 3D shape is used for computing the correspondence between the projecive images at each camera position. Morphing technique is used for image rendering.

In this example, a man walks toward a sofa and sits on it. After sitting on the sofa, the sofa suddenly tumbles down and he also falls down. Accidentally, a soccer ball is flying toward him. He nicely catches the ball.
 

For the technical details,

Appearance-Based Virtual View Generation of Temporally-Varying Events from Multi-Camera Images in the 3D Room , Computer Science Technical Report, CMU-CS-99-127, April 1999. ( Post Script) ( PDF )

Shigeyuki Baba