Coplanar Shadowgrams for Acquiring Visual Hulls of Intricate Objects

Acquiring 3D models of intricate objects (like tree branches, bicycles and insects) is a hard problem due to severe self-occlusions, repeated thin structures and surface discontinuities. In theory, a shape-from-silhouettes (SFS) approach can overcome these difficulties and use many views to reconstruct visual hulls that are close to the actual shapes. In practice, however, SFS is highly sensitive to errors in silhouette contours and the calibration of the imaging system, and therefore not suitable for obtaining reliable shapes with a large number of views. We present a practical approach to SFS using a novel technique called coplanar shadowgram imaging, that allows us to use dozens to even hundreds of views for visual hull reconstruction. Here, a point light source is moved around an object and the shadows (silhouettes) cast onto a single background plane are observed. We characterize this imaging system in terms of image projection, reconstruction ambiguity, epipolar geometry, and shape and source recovery. The coplanarity of the shadowgrams yields novel geometric properties that are not possible in traditional multi-view camera-based imaging systems. These properties allow us to derive a robust and automatic algorithm to recover the visual hull of an object and the 3D positions of light source simultaneously, regardless of the complexity of the object. We demonstrate the acquisition of several intricate shapes with severe occlusions and thin structures, using 50 to 120 views.

Publications


"Coplanar Shadowgrams for Acquiring Visual Hulls of Intricate Objects"
Shuntaro Yamazaki, Srinivasa G. Narasimhan, Simon Baker, and Takeo Kanade
Proc. of IEEE International Conference of Computer Vision (ICCV),
Oct, 2007.
[PDF]

"On Using Shadowgrams for Visual Hull Reconstruction"
Shuntaro Yamazaki, Srinivasa G. Narasimhan, Simon Baker, and Takeo Kanade
Technical Report CMU-RI-TR-07-29,
Aug, 2007.
[PDF]

Pictures

Projecttion Model of Coplanar Shadowgram Imaging:
A point source illuminates the object and its shadow cast on a planar rear-projection screen represents the silhouette of the object. Coplanar shadowgrams from multiple viewpoints are obtained by translating the light source. Note that the relative transformation between the object and the screen remains fixed across different views.
Experimental Setup:
The setup used to capture coplanar shadowgrams includes a digital camera, a single point light source, and a rearprojection screen. The object is placed close to the screen to cover a large field of view. Two or more spheres are used to estimate the initial light source positions.
Shadowgrams:
Example shadowgrams obtained using the setup.
Sensitivity of Visual Hull Reconstruction:
The top row shows the visual hulls reconstructed using the light source positions estimated by our method. As the number of silhouettes increases, the visual hull gets closer to the actual shape. The bottom row shows the reconstructions obtained from slightly erroneous source positions. As the number of views increases, the error worsens significantly.

Videos

(Video Result Playlist)
compressed using DivX codec
Shadowgram Acquisition:
This video demonstrates the acquisition of shadowgrams. The point source is moved within a half space with respect to a rear-projection screen, while the object, the camera and the screen all remain stationary. In order to cover the entire surface area of an intricate shape, we must capture a sufficiently large number of shadowgrams.
Acquired Shadowgrams:
This video shows a sequence shadowgrams acquired using our experimental setup. The light source positions are estimated using the shadowgrams of at least two spheres. The acquired shadowgrams can then be used to recover the visual hull of the object.
Optimization of Light Source Positions:
This video shows an optimization of light source positions using epipolar constraints and silhouette consistency. The ground truth and estimated source positions are presented respectively in red and yellow.
Optimization of Silhouette Consistency:
This video illustrates the improvement of silhouette consistency. An acquired silhouette and that generated from a reconstructed visual hull are shown respectively in green and yellow. The match between the acquired and re-projected silhouettes increases until it becomes almost perfect.
Robust Reconstruction of Visual Hull:
This video is the trace the progression of our optimization algorithm for this object starting from the erroneous reconstruction.
Reconstructed Visual Hulls:
This video is a CG rendering of the visual hull model acquired by coplanar shadowgrams.
Example Application to Computer Graphics:
This video is an example CG animation rendered using the 3D shape models acquired by coplanar shadowgrams.
ICCV 2007 Supplementary Video:
This video is a compilation of the main results of this project.

Data and Downloads


Coming Soon!