Title: Learning State Features from Policies to Bias Exploration in Reinforcement Learning Abstract: When successful on a single problem, reinforcement learning converges to an optimal policy that maps each state to the best action to take in that state. Unfortunately, this policy learning does not provide the reason why an action should be chosen at a state. In other words, policy learning in general does not map particular state features to particular actions. Therefore, it is infeasible to directly apply the learned policy to new problems that may have the same state features. However, if we use reinforcement learning to solve many individual problems in a specific domain, we can accumulate many policy functions, which can constitute a valuable source of information about the common state features. In this talk, I will present our approach and results exploring this research line. For a particular domain, we gather the policies learned by reinforcement learning in a variety of problems. We characterize each state with a set of features and use the policy values learned from featured-states as training examples to a classifier. The learned classifier represents a generalization over all the seen problems, mapping state features to actions. Policy learning in new complex problems is then improved by using the output of the classifier to bias exploration. We present different bias techniques and show empirical results that validate our approach. We demonstrate that state features are effectively learned from learned policies and that reinforcement learning in new problems explores fewer states by using the learned classifier than if no exploration bias is used.