Eyes (and fingers) on the road

  • A steering wheel developed by a CMU-led team of researchers provides real-time feedback for sleepy or distracted drivers
By Ken Chiacchia

The accident didn't just seem to happen in slow motion. The elderly driver came at me across the crowded parking lot so slowly I could count the dents in the grill of his big, torpedo-nosed car. Sandwiched between a bus and another car, I had no place to go and nothing to do but lean on my horn.

In agonizing languidness, I watched him plow into my Civic.

Based on his confusion after the crash, he probably should have stopped driving years before. But I can sympathize with his failure to do so. Seniors who can't drive face social isolation, difficulties in acquiring medical care, and increasingly must rely on friends, relatives and formal support systems.

SeungJun Kim, a systems scientist at Carnegie Mellon's Human-Computer Interaction Institute, wants to keep seniors on the road--safely. 

Much of his work has focused on helping seniors interact with their cars in a way that helps them overcome the cognitive and sensory declines of aging.

Recent press coverage has centered on Kim's work in the role played by GPS displays in distracting and confusing drivers, particularly older drivers. Along with Kevin Li of AT&T and his colleagues, Kim has developed a so-called "haptic" steering wheel that tries to overcome the confusion by giving drivers tactile, physical cues as to which way they should turn. The wheel vibrates to warn a driver when he or she is veering into trouble.

"Haptic" comes from an ancient Greek word "haptikos," meaning "I touch." Haptic communication, for instance, refers to any non-verbal communication transmitted through the sense of touch--kissing, hugging, shaking hands. Haptic technologies are those that transmit information through forces, vibrations and other sensations delivered to the hands.

Haptic steering wheels themselves aren't new. Racing video games use them; so do some luxury automakers as part of early-warning systems that vibrate the wheel to tell drivers when they're traveling off the road. But what Kim and the other investigators have done--actually guiding the movements of the drivers--hadn't been tried before.

The researchers studied 33 volunteers--16 people between ages 19 and 36, 17 people between 65 and 91--in a driving simulator that offered GPS-style navigation instructions along with simulated obstacles such as unexpected pedestrians in the road. The researchers varied the visual, auditory and haptic-wheel cues to see what combinations worked best for younger and older drivers, measuring their reactions by monitoring their eye movements, heart rate and galvanic skin response. 

They found that the haptic wheel improved performance by all drivers, reducing accidents and keeping them focused. The group presented their results at the Pervasive 2012 conference in Newcastle, U.K., in June. 

"Younger drivers preferred visual plus auditory" input, Kim says--in other words, they liked the usual GPS format of a map display plus a voice saying when a turn is coming up or must be made. Older drivers preferred the combination of GPS-style commands and maps plus guidance from the haptic steering wheel.

But--surprise--the combinations that drivers preferred were actually the worst ones for them.

Though they didn't particularly like the haptic wheel, younger drivers actually did better with the full combination of visual, auditory and haptic feedback. By contrast, though older drivers liked the visual maps, the maps in fact appeared to distract them--and giving them only auditory and haptic feedback helped reduce the amount of forethought they needed and improved the safety of their driving.

Kim speaks of "cognitive distance"--the gap between the "physical space" (in other words, the real world) and the "virtual information spaces" we must correlate with it. While understanding a two-dimensional map seems simple, it actually takes a measurable amount of time and brain power to translate the information into three dimensions. The process of aging affects both the amount of time it takes us to read a map--reflecting "spacial cognition"--and our ability to divide our attention between the map and the road.

With colleagues at Carnegie Mellon and the Neowiz Lab in Korea, Kim has studied what factors make car dashboard displays easier and quicker to understand for both older and younger drivers. They soon found that in-dash displays, such as those built into more expensive cars, are the most difficult to manage because they're too far away from the windshield, requiring drivers to spend too much time with their eyes off the road. 

Kim and Anind K. Dey, an associate professor at CMU's Human-Computer Interaction Institute, wondered if a so-called "augmented reality," or "AR," display, combining real and virtual images projected on the inside of the windshield, might be better. They created a 3D-virtual driving simulator, something like a video game, that could display either a mixed realistic road view with a cartoon-like "GPS" map, or a more traditional combination of windshield and dashboard-mounted display. 

They tested the two systems on 12 seniors 65 and older, and 12 younger drivers between 19 and 41.

The AR display improved the seniors' performance measurably, reducing their missed turns and pedestrian accidents each by more than 50 percent. While the younger drivers didn't show a significant difference in performance between the AR and traditional displays, tracking their eye movements showed that they were less distracted by the AR display.

Kim's work has introduced a new, useful concept to the field of driver interfaces, says Ian Oakley, an assistant professor at the University of Madeira in Portugal and an adjunct assistant professor at CMU's HCI Institute. Oakley's own expertise is in haptic feedback for user interfaces.

"There's a longstanding acknowledgment that [haptic] technology will be useful," Oakley says. "I think [Kim's work] is a really neat, very appropriate way of intervening: putting the feedback inside the actual controls," and so making the feedback more instantly intuitive. "I think that's one of the main contributions here."

Jesse Hoey, an assistant professor at the University of Waterloo, Ontario, comes to the same conclusion, but from a different perspective: he uses computer vision and artificial intelligence to create adaptive technologies for Alzheimer's disease.

"One of the most important things in helping elderly people as they age ... is to help people stay mobile," Hoey says. "This turns out to be very important for [their] wellbeing."

"One of the slightly controversial aspects [of Kim's work] is that the tests that they've been doing are all on a simulator," he adds. "[Some] people might say, 'All you've done is shown your haptic feedback works on this kind of simulator.' Although this is true in the long run, you have to start somewhere." He feels that the work is a necessary first step before moving to road tests, "which is likely to be an expensive thing to do."

Testing is important to verify the results, Oakley says, because so much of the work depends on a concept of information overload that has come into question from some quarters. The idea is that "using multiple sensory channels can transmit more information to a person without an increase in overload," he says. "But these are theories rather than facts ... and recent results indicate they may not be entirely accurate ... There are central-processing limits to attention." But he does think Kim's work will find practical application.

"I think this technology is a help ... situational awareness has to be focused and immediate. It's really about removing irrelevant information."

-- Ken Chiacchia is a Butler County, Pa.-based science and technology writer whose work regularly appears in the Pittsburgh Post-Gazette. A past winner of the Carnegie Science Center journalism award, Chiacchia has written both non-fiction and science fiction and is a member of both the National Association of Science Writers and the Science Fiction and Fantasy Writers of America.

For More Information: 
Jason Togyer | 412-268-8721 | jt3y@cs.cmu.edu