Sorting Computational Sensor
Experimental Data and Demos

Computational Sensor Laboratory
The Robotics Institute
Contact: brajovic@cs.cmu.edu

Adapting to Deliver Environmental Information

The high dynamic range scenario is created in an office setting with a back illuminated subject. A CCD camera has a limited dynamic range and images the subject as a silhouette. The sorting computational sensor, on the other hand, adjusts its dynamic range to view both bright and dark regions. In fact, the sorting image sensor provides more per pixel information than can be shown here in 8-bit-per-pixel image. 

*The sorting sensor is an early 21x26-pixel prototype

To overcome the 8-bit limitation and aid the presentation, both images are segmented into bright and dark regions. The bright regions are “painted” black, while the raw data of the dark region are amplified by 2. With this transformation additional bits provided by the sorting sensor become visible. 

Both CCD and sorting sensor images underwent this  processing.  CCD wasn't providing information in the face and any post processing is useless.

go back...

Adapting to High Contrast

This is a linear (natural) image reconstructed from the image of indices and the cumulative histogram curve both provided by the sorting sensor.  (Again to display in 8-bits we had to throw away some information.) Clearly, all scene details are visible.

You can view a movie.

*The sorting sensor is a more recent 128x128-pixel prototype.

A CCD camera cannot cope with dynamic range variation even with an automatic gain control which was active during this experiment. Clearly missing or corrupt sensory information inevitably leads to unreliable system performance; today's cameras are the major culprit.

You can view a movie.

go back...

Adapting to Low Contrast

The slight shading on the background wall is an example of low-contrast scene; it spans a few gray levels in the CCD camera (top image).  This low contrast is not only greatly enhanced in the image of indices (bottom image), but the new intermediate levels are measured. 

The CCD camera was adjusted to image general scene content inlcuding the bright computer monitor. The wall shading spanned only a few gray levels.

Since the sorting sensor automatically adapts to the scene content, it encoded as many gray levels in the wall area as there are pixels. 

While there are several alternative techniques to cope with high dynamic range using conventional sensors (i.e., multiple integration periods, or multiple attenuated images), there is no way to cope with low contrast scenes, especially when the range of possible values is not known in advance.  Due to its self-adaptive behavior, the Sorting Sensor handles it automatically.

go back...

Adaptive Behavior

See the animation of this sequence to see the adaptive behavior of the sensor.  

 

 

 

 

go back...