Teacher: Emma Brunskill

TA: Christoph Dann

Time and location: Mon and Wed at 1:30-2:50, GHC 4101

Office hours: TBA and by appointment.

Overview

To realize the dreams and impact of AI requires creating autonomous systems that can learn to make good decisions.

In this advanced topics in AI class, we will start with a short background in reinforcement learning and sequential decision making under uncertainty. We will then quickly move on to covering state-of-the-art approaches for some of the critical challenges in applying reinforcement learning to the real world (e.g. robotics, computational sustainability, personalized education and healthcare). These challenges include leveraging old data to make new decisions, highly sample efficient RL, and safe and risk sensitive RL.

Prerequisites: The course is mainly intended for graduate students in computer science, machine learning, and robotics. Undergraduates and students in other relevant areas are welcome to join if you have the relevant background. We will assume basic familiarity with ideas in probability, machine learning, and control/decision making, and programming skills (which could include Matlab). It is useful, but not required, to have taken one or more of the following classes (or their equivalent): Machine Learning, Statistical Techniques in Robotics, and Artificial Intelligence. Creativity and enthusiastic participation are required.

Requirements

Grades are based on homeworks (30%), a midterm (20%), a final project (40%), and participation (10%).

Late Policy

You will have 4 late days without penalty to be used across the entire semester. These can only be used for homeworks, not the project deliverables. After those late days are used, you will be penalized according to the following policy: (1) homework is worth full credit at the start of class on the due date; (2) homework is worth half credit for the next 48 hours; (3) it is worth zero credit after that.

Collaboration

Unless otherwise specified, written homeworks can be discussed with others but must be written up individually. You must write on your homework the student names you collaborated with.

Piazza

We will use the Piazza course management system.