Webcam Clip Art: Appearance and Illuminant Transfer from Time-lapse Sequences

Teaser
We introduce a new, high-quality dataset of calibrated time-lapse sequences (a). Illumination conditions are estimated in a physically-consistent way and HDR environment maps are generated for each image (b). This database can be used as a ``clip art'' library for applications like inserting lighting-consistent 2D objects, such as the buildings in (c), or relighting 3D objects, such as the statue in (d).

Dataset
Calibrated webcams

Webcams placed all over the world observe and record the visual appearance of a variety of outdoor scenes over long periods of time. The recorded time-lapse image sequences cover a wide range of illumination and weather conditions -- a vast untapped resource for creating visual realism. In this work, we propose to use a large repository of webcams as a "clip art" library from which users may transfer scene appearance (objects, scene backdrops, outdoor illumination) into their own time-lapse sequences or even single photographs. The goal is to combine the recent ideas from data-driven appearance transfer techniques with a general and theoretically-grounded physically-based illumination model. To accomplish this, the paper presents three main research contributions: 1) a new, high-quality outdoor webcam database that has been calibrated radiometrically and geometrically; 2) a novel approach for matching illuminations across different scenes based on the estimation of the properties of natural illuminants (sun, sky, weather and clouds), the camera geometry, and illumination-dependent scene features; 3) a new algorithm for generating physically plausible high dynamic range environment maps for each frame in a webcam sequence.

Publications

Paper thumbnail "Webcam Clip Art: Appearance and Illuminant Transfer from Time-lapse Sequences"
Jean-François Lalonde, Alexei A. Efros, and Srinivasa G. Narasimhan.
ACM Transactions on Graphics (SIGGRAPH Asia 2009), 28(5),
December 2009.
[PDF] [BibTeX]

Video

(Video Result Playlist)

Webcam Clip Art from Jean-Francois Lalonde on Vimeo.

Snapshot for movie [H264, 720x480 NTSC, 63.9MB]

Talk


Download the slides from the talk given at SIGGRAPH Asia 2009 in the following formats:


[Apple Keynote, 90.2MB], original version used at SIGGRAPH Asia 2009;
[MS Powerpoint, 76.2MB], export from Apple Keynote;
[PDF, 50.8MB], different stages of animations shown.

Dataset


Find the data associated with this paper here.

Code


Here are some software packages relevant to that project:


- Calibration from sky
- Sky model

Funding


This research is supported by:


- NSF CCF-0541230
- NSF IIS-0546547
- ONR N00014-08-1-0330
- NSF IIS-0643628
- Microsoft Research Fellowship
- Microsoft Research grant

Copyright notice


The documents contained in these directories are included by the contributing authors as a means to ensure timely dissemination of scholarly and technical work on a non-commercial basis. Copyright and all rights therein are maintained by the authors or by other copyright holders, notwithstanding that they have offered their works here electronically. It is understood that all persons copying this information will adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder, except when identified by Creative Commons License 2.0, in which case the license applies to both the original and modified versions of the images.