.: Main :: How it Works :: Presentations :: Publications :: Posters :: Acknowledgements :.

Toward a Smart Automotive Headlight for
Seeing Through Rain and Snow


Frequently Asked Questions




What is the motivation behind the smart headlight?
Nearly half of all accidents occur at night despite 25% less traffic on the road due to limited visibility. Visibility is further reduced during rain and snow storms creating extremely dangerous driving conditions. A fourteen-year average of accident statistics from the National Highway Safety Transportation Administration indicates that 900,000 accidents, 400,000 injuries, and 4,000 fatalities occur in rain and snow. We aim to assist drivers during evening and night storms by improving driver visibility and reducing stress by reducing the visibility of illuminated precipitation particles (e.g., raindrops, snowflakes, etc.).

back to top

What exactly is a smart headlight?
Standard vehicle headlights illuminate everything in their pathway, which is necessary for nighttime driving. However, when it is raining or snowing the headlights also illuminate the falling precipitation particles (e.g., raindrops, snowflakes, etc.) causing bright distracting streaks to the driver. The smart headlight addresses this issue by quickly controlling individual light rays from the headlight to avoid particles. By drastically reducing backscattered light, the smart headlight makes precipitation particles disappear right in front of the driver.

back to top

How did you come up with this idea?
Our lab has been researching visibility enhancement in poor weather conditions for more than a decade. We have developed tools for removing fog, haze, mist, rain, snow, dust and murky waters from images and videos. But, we thought, wouldn't it be great if the driver sees better with her/his eyes? That thought started this project a couple of years ago. Along the way, we took a slight detour and developed the first 3D display with water drops.

back to top

How will the smart headlight work?
The idea behind the smart headlight is to integrate an imager and processing unit with a light source. The imager captures images illuminated by the light source and the processing unit detects precipitation particles in the upper part of the image, predicts the future location of the particles, then controls the light source to illuminate between particles.

back to top

What is the current stage of development for the smart headlight?
We have conducted computer simulations to show that the idea is feasible, and built a prototype system in the laboratory. A stationary prototype system at 120 Hz has been tested with a rain generator. The result is a dramatic reduction in visibility of illuminated rain drops. Currently, we are investigating making the system faster and more compact in order to test on a moving platform.

back to top

What are the components of the prototype system?
The current prototype consists of a camera with gigabit ethernet interface (Point Grey, Flea3), DLP projector (Viewsonic, PJD62531), and desktop computer with Intel architecture (Intel i7 quad core processor).

back to top

Why is a beamsplitter used?
The beamsplitter (50/50) permits optically co-locating the camera and projector to eliminate the need for stereo reconstruction thus reducing computations and increasing system speed. The downside is that 50% of light from the projector is lost making drop detection more difficult due to decreased contrast.

back to top

Why not post-process a video from a camera (visible or IR) on a car to remove rain?
This is a perfectly plausible solution. For example, see our previous work of removing rain, snow, and fog from videos. Similarly, other systems based on thermal imaging exist to help see through rain and fog. However, the resulting videos need to be displayed somewhere in the car. This can be distracting for the driver; he or she needs to look away and refocus on the small display in a difficult situation. The proposed system enhances the visibility for the driver's naked eye and also helps the drivers in oncoming vehicles.

back to top

How about using gated imaging with a fast pulsed light source as headlight?
Yes, time-gated imaging is a very good way to block these reflections at any particular distance. The issues here though are: it may block anything else at that distance and it may not block reflections from many distances. We also did not want to gate the windshield of a car (far more expensive) or have drivers wear shuttered glasses that are synched with the headlights (more cumbersome). We are interested in having the driver see better with his/her naked eye. Same reasoning goes for methods that capture videos, post-process them to remove the rain/snow and display them in the car's display area.

back to top

What are the technical challenges?
Because the prototype was built with off-the-shelf components, data transfer speed is slower than if the components were more closely integrated (e.g., in an embedded system). The refresh rate of the projector and camera sensitivity for short exposure times also have an effect on system performance.

In order for the system to be used on an automobile it needs to be made faster and more compact. Improvements in speed and size can be made by developing specialized hardware that more closely integrates the camera and DLP projector with a processing unit (e.g., DSP). More sophisticated algorithms will be needed to maximize system speed and account for vehicle motion, wind turbulence, etc.

back to top

What is the speed of the prototype system?
The current system runs at 120 Hz. The camera uses a 5 ms exposure time and the system has a total latency of 13 ms; 4 ms for transferring data from camera to computer, 5 ms for drop detection and prediction, and generation of the projection image, and 4 ms for transferring data from the computer to the projector.

back to top

How fast will the system need to be for use in vehicles?
Computer simulations show that a system operating near 1,000 Hz, with a total system latency of 1.5 ms, and exposure time of 1 ms can achieve 96.8% accuracy with 90% light throughput during a heavy rainstorm (25 mm/hr) on a vehicle traveling 30 km/hr. However, 400 Hz with less accuracy will be a significant (>= 70%) improvement for the driver.

back to top

What challenges remain?
In order for the system to be used on an automobile it needs to be made faster and more compact. Improvements in speed and size can be made by developing specialized hardware that more closely integrates the camera and DLP projector with a processing unit (e.g., DSP). More sophisticated algorithms will be needed to maximize system speed and account for vehicle motion, wind turbulence, etc.

back to top

Are there any concerns with deploying such a headlight?
Since people heavily depend on headlights for driving at night, the smart headlights must maintain adequate illumination of the environment. We refer to this as light throughput. A standard headlight has 100% light throughput. An extensive user study will be conducted to determine the necessary accuracy for reducing driver stress.

back to top

How long until the smart headlight can become a product?
The technology currently being researched for building a faster, compact system is expected to take 3 to 4 years to complete. Commercializing it as a product will take additional years.

back to top
Are there other applications for the smart headlight?
We believe that the technology behind the smart headlight has an array of different applications. Besides rain and snow, the smart headlight can be used to draw the driver's attention to obstacles such as wildlife, pedestrians, roadblocks, lanes, lane markings and dividers. It can also be used to avoid blinding the oncoming drivers.

At its core, the headlight is a fast reactive imaging and illumination/display device and designs with different form factors be used in many other domains including consumer mobile applications.

back to top

back to project