NSF CAREER: Making Computer Vision Successful in Scattering Media

Principal Investigator: Srinivasa G. Narasimhan
Carnegie Mellon University


In recent years, computer vision has seen significant advances in the core areas of image sensing and interpretation. This success has resulted in great demand for vision techniques in application domains ranging from intelligent transportation and security to oceanography (underwater imaging), to astronomy (telescope and satellite imaging), to even biology and medical systems (microscopic and medical imaging). Unfortunately, there is one fundamental hurdle that can stop vision from having successful impact in these areas --- the assumption that light propagates in a transparent medium (pure air) without any alteration. Thus, today vision systems fail to perform in the presence of light scattering by a wide range of particulate media, such as bad weather (fog, mist, haze, snow, rain), murky water, smoke, dust, smog and biological tissue.

This research is devoted to making computer vision successful in scattering media. In computer vision, image formation has been defined as "a geometric mapping from the 3D world to the 2D image", which inherently leads to loss of information. The PI strongly argues that light scattering must not be viewed as "noise" that a traditional vision algorithm needs to overcome, but rather as a new form of "encoding" of light and hence, the images themselves. The key idea then is to derive a series of compact physically based analytic (or semi-analytic) models for light transport to represent image formation in scattering media. These models encode the "lost third dimension" back into images. The analytic forms of the models --- though not as elaborate as the slow simulations in computational physics --- are accurate enough to model the aggregate scattering effects in images and thus make it possible to invert light transport. The inverse light transport methods will then be applied in conjunction with traditional vision algorithms to match their performances in clear air.

The results from this research will have broad and long-term impact across a wide variety of domains. The (semi-)automatic intelligent transportation systems that assist drivers in navigation will be able operate in common bad weather conditions such as fog, snow and rain, indeed when they are most required. Similarly, field robots will navigate better in hazardous environments such as smoke and dust. Underwater exploration, safety, and rescue tasks can be made possible in murky under water conditions. Understanding optical properties of tissues can assist doctors in medical diagnosis of tumors and cancers. Finally, the derived models can be used to also add realistic effects of scattering to imagery for digital entertainment (movies and video games), scientific education and training.

For details on the various projects developed under this grant, please visit the Illumination and Imaging Laboratory at Carnegie Mellon University.

First International Symposium on Volumetric Scattering in Vision and Graphics, 2007.