16-899 Adaptive Control and Reinforcement Learning (Fall 2020)
(Last Update: 12/28/2020)
Time: Tuesday and Thursday 8:00-9:20
Location: zoom
Instructor: Changliu Liu, cliu6@andrew.cmu.edu
Instructor office hours: by appointment
Teaching Assistant: Charles Noren, cnoren@andrew.cmu.edu
TA office hours: Friday 12:30-13:30
Canvas: https://canvas.cmu.edu/courses/18691
Course Description
This course will discuss algorithms that learn and adapt to the environment. This course is directed to students—primarily graduate although talented undergraduates are welcome as well—interested in developing adaptive software that makes decisions that affect the world. This course will discuss adaptive behaviors both from the control perspective and the learning perspective.
Key Topics
optimal control, model predictive control, iterative learning control, adaptive control, reinforcement learning, imitation learning, approximate dynamic programming, parameter estimation, stability analysis.
Course Goal
To familiarize the students with algorithms that learn and adapt to the environment. To provide a theoretical foundation for adaptable algorithm.
Syllabus
Prerequisite
As an advanced course, familiarity with basic ideas from control theory, robotics, probability, machine learning will be helpful. Useful courses to have taken in advance include Statistical Techniques in Robotics, Artificial Intelligence, and Kinematics, Dynamics, and Control. As the course will be project driven, prototyping skills including C, C++, Python, and Matlab will also be important. Creative thought and enthusiasm are required.
Assessment Structure
Homework - 30%
Quizzes - 20%
Final project - 10% (proposal) + 30% (report) + 10% (presentation)
Student Final Projects (Selected)
Alex Wu and Wei Liang.
Comparison between Model Predictive Control and Iterative Learning Control on Autonomous Race Tracking.
Gaurav Pathak and Swaminathan Gurumurthy.
Comparing Policy Optimization approaches with learnt models in the context of Offline learning.
[Slides]
Haowen Shi.
Scanning Trajectory Planning for Confined Space Inspection.
[Slides]
Jielin Qiu, Weiye Zhao, and Rui Chen.
Model-based Reinforcement Learning for Autonomous Driving in Dynamically Varying Environments.
[Slides]
References
Optional textbook:
Goodwin. Adaptive Filtering, Prediction, and Control.
Bertsekas. Optimal Control and Reinfocement Learning.
Anderson and Moore. Optimal Control, Linear Quadratic Methods.
Borrelli. Predictive control for linear and hybrid systems.
Sutton and Barto. Reinforcement Learning: An Introduction.