Who: Will Uther What: Automated Acquisition of Hierarchical Decomposition Graphs for RL When: Tuesday Jan 5th 12:00 - 1:00 Where: Wean 8220 Why: Cause I need the practice :) Abstract: Automated Acquisition of Hierarchical Decomposition Graphs for RL Reinforcement Learning (RL) is well understood for small, discrete, fully observable domains, but for real-world domains classical solutions are too slow to be practical. Efforts based on function approximation have allowed RL to scale to some real-world domains, such as factory scheduling, but others, like real-time robot games, remain out of reach. These previous function approximation efforts, referred to as state abstraction techniques, have focused on generalizing over local regions of the state space. I will investigate new forms of function approximation which generalize over similar solution structure in different areas of the state space, often referred to as temporal abstraction. In particular, I am interested in the automatic formation of hierarchical decomposition graphs for RL in continuous state spaces. To form a hierarchical decomposition I propose to automatically detect, and generalize across, common substructure within a symbolic function approximation system such as a decision tree. I have devised a new tree-based hierarchical RL approach that supports sharing and reuse of learned information, and I will continue to investigate effective means of discovering common substructure in tree-based approximators. I propose to investigate the reuse of learned information both within and between tasks through the development of a set of pedagogic domains that allow accurate empirical evaluation of my algorithm. I will also evaluate my approach on a large, continuous state space, real-world task.