Better safe than sorry? Neural prediction errors reveal a risk-sensitive reinforcement learning process

Abstract:

Which of these would you prefer: getting $10 with certainty or tossing
a coin for a 50% chance to win $20? Whatever your answer, you probably
were not indifferent between these two options. In general, human
choice behavior is influenced not only by the expected reward value of
options, but also by their variance, with people differing in the
degree to which they are risk-averse or risk-seeking. Economic,
psychological and neural aspects of this are well studied when
information about risk is provided explicitly. However, we must
normally learn about outcomes from experience, through trial and
error. Traditional reinforcement learning (RL) models of action
selection, however, rely on temporal difference methods that learn the
mean value of an option, ignoring risk.  We used fMRI to test this
assumption by examining the neural correlates of reinforcement
learning and asking whether they are indeed indifferent to risk. Our
results show that reinforcement learning is modulated by experienced
risk, and reveal a close coupling between the fluctuating,
experience-based, evaluations of risky options measured neurally, and
fluctuations in behavioral choice. This suggests that risk sensitivity
is integral to human learning, illuminating economic models of choice
and neuroscientific models of learning.

Joint work with: Jeffrey A. Edlund, Peter Dayan, John P. O'Doherty

Bio: 

Yael is an assistant professor at the Princeton Neuroscience Institute
(PNI) and the Psychology Department at Princeton University since
September 2008. She was also a postdoc at Princeton, and earned her
PhD at The Hebrew University of Jerusalem (Israel) while conducting
most of her research at the Gatsby Computational Neuroscience Unit
(UCL, London). Her research focuses on normative computational models
of learning and decision making, and in understanding the neural basis
for simple day to day trial and error learning.