What Do the Sun and the Sky Tell Us About the Camera?

Camera parameters automatically estimated from the sky appearance.
(webcam sequences from the AMOS database)

Sky/cloud segmentation

As the main observed illuminant outdoors, the sky is a rich source of information about the scene. However, it is yet to be fully explored in computer vision because its appearance in an image depends on the sun position, weather conditions, photometric and geometric parameters of the camera, and the location of capture. In this paper, we analyze two sources of information available within the visible portion of the sky region: the sun position, and the sky appearance. By fitting a model of the predicted sun position to an image sequence, we show how to extract camera parameters such as the focal length, and the zenith and azimuth angles. Similarly, we show how we can extract the same parameters by fitting a physically-based sky model to the sky appearance. In short, the sun and the sky serve as geometric calibration targets, which can be used to annotate a large database of image sequences. We use our methods to calibrate 22 real, low-quality webcam sequences scattered throughout the continental US, and show deviations below 4% for focal length, and 3 degrees for the zenith and azimuth angles. Once the camera parameters are recovered, we use them to define a camera-invariant sky appearance model, which we exploit in two applications: 1) segmentation of the sky and cloud layers, and 2) data-driven sky matching across different image sequences based on a novel similarity measure defined on sky parameters. This measure, combined with a rich appearance database, allows us to model a wide range of sky conditions.


Paper thumbnail "What Do the Sun and the Sky Tell Us About the Camera?"
Jean-François Lalonde, Alexei A. Efros, and Srinivasa G. Narasimhan.
International Journal on Computer Vision, 88(1):24-51,
May 2010.
[PDF] [BibTeX]
(original publication is available at www.springerlink.com)
Paper thumbnail "What Does the Sky Tell Us About the Camera?"
Jean-François Lalonde, Alexei A. Efros, and Srinivasa G. Narasimhan.
European Conference on Computer Vision,
[PDF] [BibTeX]

Poster - ECCV 2008

Poster thumbnail Download the poster that was presented at ECCV 2008 here: [PDF, 11.6MB]


(Video Result Playlist)
Original sequences (1 day):
257-img 466-img 601-img
Corresponding camera parameters and sun position:
257-plot 466-plot 601-plot
All movies are in MPEG-4 format.

Additional results

Camera parameters and corresponding sun positions.


You can find software packages relevant to that project HERE.


This research is supported by:

- ONR N00014-08-1-0330
- NSF IIS-0643628
- NSF CCF-0541307
- NSF CCF-0541230

Copyright notice

The documents contained in these directories are included by the contributing authors as a means to ensure timely dissemination of scholarly and technical work on a non-commercial basis. Copyright and all rights therein are maintained by the authors or by other copyright holders, notwithstanding that they have offered their works here electronically. It is understood that all persons copying this information will adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder, except when identified by Creative Commons License 2.0, in which case the license applies to both the original and modified versions of the images.