Who: Will Uther
What: Automated Acquisition of Hierarchical Decomposition Graphs for RL
When: Tuesday Jan 5th 12:00 - 1:00
Where: Wean 8220
Why: Cause I need the practice :)
Abstract:
Automated Acquisition of Hierarchical Decomposition Graphs for RL
Reinforcement Learning (RL) is well understood for small, discrete,
fully observable domains, but for real-world domains classical solutions
are too slow to be practical. Efforts based on function approximation
have allowed RL to scale to some real-world domains, such as factory
scheduling, but others, like real-time robot games, remain out of
reach. These previous function approximation efforts, referred to as
state abstraction techniques, have focused on generalizing over local
regions of the state space. I will investigate new forms of function
approximation which generalize over similar solution structure in
different areas of the state space, often referred to as temporal
abstraction. In particular, I am interested in the automatic formation
of hierarchical decomposition graphs for RL in continuous state spaces.
To form a hierarchical decomposition I propose to automatically detect,
and generalize across, common substructure within a symbolic function
approximation system such as a decision tree. I have devised a new
tree-based hierarchical RL approach that supports sharing and reuse of
learned information, and I will continue to investigate effective means
of discovering common substructure in tree-based approximators. I
propose to investigate the reuse of learned information both within and
between tasks through the development of a set of pedagogic domains that
allow accurate empirical evaluation of my algorithm. I will also
evaluate my approach on a large, continuous state space, real-world
task.