June 17, 3:00, WeH 4601 Hidden State and Short-Term Memory Organizer: Lonnie Chrisman, lonnie.chrisman@cs.cmu.edu Speakers: Lonnie Chrisman & Michael Littman, CMU Many realistic agents cannot directly observe every relevant aspect of their environment at every moment in time. Such hidden state causes problems for many reinforcement learning algorithms, often causing temporal differencing methods to become unstable and making policies that simply map sensory input to action insufficient. In this session we will examine the problems of hidden state and of learning how to best organize short-term memory. I will review and compare existing approaches such as those of Whitehead & Ballard, Chrisman, Lin & Mitchell, McCallum, and Ring. I will also give a tutorial on the theories of Partially Observable Markovian Decision Processes, Hidden Markov Models, and related learning algorithms such as Baum-Welsh/EM as they are relevant to reinforcement learning.