Lifelogging Memory Aid for People with Alzheimer's Disease

Matthew Lee & Anind Dey
matthew.lee [at]

[ Problem | Goals | Questions | Solution | Papers and Posters | Collaboration | Acknowledgments ]

Research Problem

People with recent episodic memory impairment lack an awareness of the actions, events, and experiences of their everyday lives. This lack of continuity in their lives and can lead to confusion, depression, a loss in their sense of self, and impaired decision-making.

One common cause of episodic memory impairment is Alzheimer's disease (AD). One of the earliest symptoms of AD and Mild Cognitive Impairment (a condition that often precedes AD) is loss of the ability to recall memories of recent experiences. About 18 million people worldwide currently have been diagnosed with AD, with that number expected to double over the next 20 years.

Supporting an individual's memories of recent experiences can increase their confidence, independence, and overall quality of life as well as decrease the burden on the individual's caregiver.


To explore and understand the socio-cognitive bases of recent episodic memory impairment associated with Alzheimer's disease and design a solution that will help increase self-awareness of recent experiences to increase independence, slow the rate of progression of decline, and reduce the burden on caregivers.

Research Questions

Are memory aids that leverage rich, contextualized media effective in compensating for recent memory impairments and reducing caregiver burden?

The most significant impact of the early stages of Alzheimer's disease is a loss of the ability to remember recent memories. Previous research (Gowans 2003) has found that using rich non-personalized multimedia was useful as a conversation aid during reminiscence therapy to re-activate very old memories. Will personalized multimedia representations of recent memory be effective in helping re-activate more recently experienced memories?

What types of experiences do people with AD most often forget? What types of experiences do people with AD most often remember?

The solution will increase awareness of memories through two forms of support: memory reconstruction for memories that the individual has completely forgotten and memory recollection for memories that need extra cues to trigger their retrieval.

How do memory breakdowns manifest themselves in practice?

Understanding the context of memory failures allows the solution to deliver memory assistance in the proper context.

How do these breakdowns affect the everyday routines?

Everyday routines provide structure and continuity, especially for demented individuals. Disruption of these routine can lead to confusion and further disruption of structured routines. As with any memory aid system, the user needs to be able to remember to use it. Thus, the solution needs to complement existing activites.

What kinds of memory cues are most effective?

The choice of memory cues is critical for facilitating accurate recollection of recent experiences. We will explore how different media (eg: imagery, sounds, text, tactile) can contribute to the richness of the memory reconstruction and recollection experience.

What kinds of memory support mechanisms exist in the home environment?

It has been observed that while many demented individuals may perform poorly on standardized memory tests, they can function well (almost at a level where others may not be able to tell they have dementia) in their home environment. We want to investigate how the home supports memory.

How does the caregiver support the memory of the individual with memory impairment?

The caregiver plays an integral role in providing physical and cognitive support for the individual with memory impairment. We want to investigate what strategies they use and to gain their perspectives on what can be an effective technological intervention to both compensate for the care recipient's memory and also to reduce the burden of providing care.

Research Methods

We are conducting an ethnography of people with Alzheimer's disease and their caregivers. The data collection instruments used include semi-structured interviews with each the person with AD and the caregiver, a contextual-inquiry where a research follows and observes the person with AD and caregiver as they go about their daily routines at home, and a diary study where the caregiver will record instances of memory failures and their context.


We designed a lifelogging system that follows a three-step process of capture, cue selection, and review to support recollection of episodic memories. Using a combination of sensors, the system passively records experiences of the users choosing. Then, the caregiver uses the CueChooser application to construct a narrative of the experience by selecting and annotating the best memory cues from the recorded lifelog data. Finally, with the MemeXerciser application, the caregiver-chosen cues are presented to the person with EMI in a way that maximizes the opportunities for them to think deeply about the cues to remember associated details on their own without requiring repeated live help from their caregiver.

Papers and Posters

M. L. Lee and A. K. Dey, "Lifelogging Memory Appliance for People with Episodic Memory Impairment ," Proceedings of UbiComp 2008, pp. 44-53. (pdf)

M. L. Lee and A. K. Dey, "Wearable Experience Capture for Episodic Memory Support," Poster abstract to appear in Proc. International Symposium on Wearable Computing. (pdf)

M. L. Lee and A. K. Dey, "Memexerciser: Lifelogging Technology for People with Episodic Memory Impairment," poster presented at NSF Site Visit to the Quality of Life Technology Center. (pdf)

M. L. Lee and A. K. Dey, "Providing Good Memory Cues for People with Episodic Memory Impairment," Proceedings of the 9th international ACM SIGACCESS conference on Computers and accessibility, pp. 131 - 138, 2007. (pdf)

M. L. Lee and A. K. Dey, "Capture and Access Lifelogging Technology for People with Episodic Memory Impairment," presented at Workshop on Intelligent Systems for Assisted Cognition. (pdf)

Lee, M.L., Dey, A.K. Capturing and Reviewing Context in Memory Aids. Presented at the Workshop on Designing Technology for People with Cognitive Impairments, CHI 2006. (pdf)

Poster: Using context as a memory aid for people with Alzheimer's disease. Presented at CMU CS50 and HCII 12th Anniversary celebration. (pdf)


Advisor: Anind K. Dey, HCI Institute, Carnegie Mellon University

Clinical Collaborators:
James T. Becker, Associate Director, Alzheimer's Disease Research Center, University of Pittsburgh
David A. Wolk, Assistant Professor, Dept of Neurology, University of Pittsburgh


This work is part of the Quality of Life Technology Center.

We would like to thank Microsoft Corporation for selecting this project to receive hardware and software grants as part of the Microsoft Research Digital Memories (Memex) 2005 RFP Awards and for additional support through the Assisted Cognition Research Program.

<< Back to Matthew Lee's Homepage

contact: matthew.lee [at]  |  updated: 17-July-2008