|
|
Foundations of Robotics
Seminar, May 16, 2007
Time
and Place | Seminar Abstract | Speaker
Appointments
Structured prediction
techniques for imitation learning
Nathan Ratliff
(presenter)
with
Dave Bradley,
Drew Bagnell, Martin Zinkevich,
and Joel Chestnutt
NSH 1507
Refreshments 4:15 pm
Talk 4:30 pm
Imitation learning of sequential, goal-directed behavior
by standard supervised techniques is often difficult. We frame learning such
behaviors as a maximum margin structured prediction problem over a space of
policies. In this approach, we learn mappings from features to cost so an
optimal policy in an MDP with these cost mimics the expert's behavior.
Further, we demonstrate a simple, provably efficient
approach to structured maximum margin learning, based on the subgradient method, that leverages
existing fast algorithms for inference. Although the technique is general, it
is particularly relevant in problems where A* and dynamic programming
approaches make learning policies tractable in problems beyond the limitations
of a QP formulation. In the context of
policy learning, we call this algorithm Maximum Margin Planning (MMP). We demonstrate our approach applied to route
planning for outdoor mobile robots, where the behavior a designer wishes a
planner to execute is often clear, while specifying cost functions that
engender this behavior is a much more difficult task.
We extend this algorithm to learning nonlinear
mappings from features to cost functions using variants of the functional
gradient descent view of boosting. These
approaches utilize simple binary classification or regression to improve
performance of MMP imitation learning, and naturally extend to the class of
structured maximum margin prediction problems.
We apply these nonlinear techniques to navigation and planning problems
for outdoor mobile robots as well as to learning heuristics and footstep
placement for legged locomotion.
For appointments, please contact Nathan Ratliff.
The Robotics Institute is part of the School of Computer Science, Carnegie Mellon University.