# Reinforcement Learning

Modern computer systems face complex decision-making tasks such as
controlling traffic signals, scheduling factory production, planning
medical treatments, allocating investment portfolios, routing data
through communications networks, and playing expert-level backgammon
or chess. Such tasks are difficult *sequential decision
problems*:
In such complex problems, optimal decision policies are in general
unknown, and it is often difficult, even for human domain experts, to
manually encode even reasonably good decision policies in software. A
growing body of research in Artificial Intelligence suggests the
following alternative methodology:

A decision-making algorithm can autonomously *learn*
effective policies for sequential decision tasks, simply by
simulating the task and keeping statistics on which decisions
lead to good ultimate performance and which do not.

The field of *reinforcement learning* defines a principled
foundation for this methodology, based on classical dynamic
programming algorithms for solving Markov Decision Problems. The
goal of reinforcement learning is to successfully approximate the
*optimal value function*, a special evaluation function that
predicts the expected long-term quality of visiting any state. For
example, the optimal value function in the game of backgammon is
defined as the probability of Player X winning at any given board
position, assuming optimal play by both players for the rest of the
game. Having an (approximately) optimal value function allows
(approximately) optimal decisions to be made.

## More Information

- Modern reinforcement learning originated in the Adaptive Networks
Laboratory at the University of Massachusetts.

UMass ANL website
- This 1996 paper surveys the field.

Kaelbling, L.P., Littman, M.L., and Moore, A.W. (1996).
Reinforcement Learning: A Survey. Journal of Artificial
Intelligence Research, Volume 4, pages 237-285.
(html), (postscript)
- The RL Repository at Michigan State contains a wealth of
information, papers and software.

`http://web.cps.msu.edu/rlr/`
- Here at CMU, there is a weekly seminar series on RL and related
topics.

RL & Friends
seminar schedule and archive

*Back to Glossary Index*