Bhiksha's Projects

Unusual Sensors

Data capture mechanisms for applications such as biometrics, speech recognition, face recognition etc. have typically employed sensors such as cameras, microphones, IR sensors etc.

In this project we explore other forms of sensing that are not traditionally associated with biometric and related applications. Specifically, we have explored Doppler Sonars to some depth; future work will look at NPR cameras, heat sensors etc.

The Doppler Effect as a Biometric Characterization

The Doppler effect is the phenomenon by which the frequencies in an EM or audio signal, as measured by an observer, change when the signal is emitted or reflected by a moving object. If the distance between a listener and the reflection/emitting object increases with time, all obeserved frequencies reduce; if the distnaces decreases with time, the frequencies increase.

If one were to bounce a high-frequency tone off a moving object, the reflected ttone would have a shifted frequency, where the frequency shift would depend on the velocity of the object. If the tone were bounced from multiple moving objects, the reflected signal would have many frequencies, one for each moving object.

The human body is an articulated object, and any movement associated with walking, gestures, talking etc. involves movements of many separate components with different velocities. The velocities of each of the components also changes with time.

If a high-frequency tone is incident on a subject, the reflected signal would would therefore have an entire spectrum of frequencies, where the spectrum is a characterization of the pattern of instantaneous velocities of all moving body parts. The combinations of velocities is characteristic of the movements, which in turn are characteristic of the subject, or the specific action that the subject is performing. The spectrum of the reflected signal (which we call the Doppler spectrum) can hence be viewed as a signature that could be used as evidence to identify the action.

The Doppler Sonar Sensor

Our Doppler Sonar comprises a tone generator that produces a 40Khz tone. The tone is reflected off the subject, and a sensor that's collocated with the tone generator captures the reflection. The entire setup can be created for about $10 in the lab.

We have employed the sonar as a sensing mechanism for gesture recognition1, recognition of talking faces 2, as a secondary sensor for speaker identification 3, gait recognition 4, and even as a UI for aiding voice activity detection5.

We are also investigating the use of the Doppler sensor as a secondary mechanism to aid speech recognition, the use of multiple sensors that permits us to account for variations in angle of approach, and machine learning methods for optimal extraction of information from the Doppler signal, and a variety of other related topics.

Collaborators

References

[1] K. Kalgaonkar and B. Raj, One-handed Gesture Recognition using Ultrasonic Doppler Sonar, IEEE Intl. Conf. on Acoustics, Speech and Signal Processing 2009.

[2] K. Kalgaonkar and B. Raj, Recognizing Talking Faces from Acoustic Doppler Measurements, IEEE Intl. Conf. on Automatic Face and Gesture Recognition, 2008.

[3] K. Kalgaonkar and B. Raj, Ultrasonic Doppler Sensor for Speaker Recognition, IEEE Intl. Conf. on Acoustics Speech and Signal Processing 2008.

[4] K. Kalgaonkar and B. Raj, Acoustic Doppler Sonar for Gait Recognition, IEEE International Conference on Advance Video and Signal-based Surveillance (AVSS2007), September 2007

[5] K. Kalgaonkar and B. Raj, Acoustic Doppler Based Front End for Hands-free Spoken User Interfaces, Proc. IEEE Spoken Language Technologies Workshop (SLT), Aruba, Dec 2006.