10703 (Spring 2018): Deep RL and Control

• Russ: Mondays 11-12pm, 8105 GHC
  

• TBD

Marking Scheme

• 3 assigments: 60%
  
• Final Project: 40%

Class goals

• Implement and experiment with existing algorithms for learning control policies guided by reinforcement, expert demonstrations or self-trials.
• Evaluate the sample complexity, generalization and generality of these algorithms.
• Be able to understand research papers in the field of robotic learning.
• Try out some ideas/extensions of your own. Particular focus on incorporating true sensory signal from vision or tactile sensing, and exploring the synergy between learning from simulation versus learning from real experience.

Resources

Books

1. [SB] Sutton & Barto, Reinforcement Learning: An Introduction
2. [GBC] Goodfellow, Bengio & Courville, Deep Learning

• Machine Learning: A Probabilistic Perspective, by Kevin P. Murphy.
• Trevor Hastie, Robert Tibshirani, Jerome Friedman (2009) The Elements of Statistical Learning
• David MacKay (2003) Information Theory, Inference, and Learning Algorithms

General references

• Szepesvari, Algorithms for Reinforcement Learning
• Bertsekas, Dynamic Programming and Optimal Control, Vols I and II
• Powell, Approximate Dynamic Programming

Online courses

• John Schulman's and Pieter Abeel's class: Deep Reinforcement Learning, Fall 2015
• Sergey Levine's, Chelsea Finn's and John Schulman's class: Deep Reinforcement Learning, Spring 2017
• Pieter Abeel's class: Advanced Robotics, Fall 2015
• Emo Todorov's class: Intelligent control through learning and optimization, Spring 2015
• David Silver's class: Reinforcement learning

Prerequisites

This course assumes some familiarity with reinforcement learning, numerical optimization, and machine learning. Suggested relevant courses in MLD are 10701 Introduction to Machine Learning, 10807 Topics in Deep Learning, 10725 Convex Optimization, or online equivalent versions of these courses. For an introduction to machine learning and neural networks, see:

• Andrej Karpathy's course
• Geoff Hinton on Coursera
• Andrew Ng on Coursera

10703 (Spring 2018): Deep RL and Control || http://www.cs.cmu.edu/~rsalakhu/10703/