10-425 + 10-625, Fall 2023
School of Computer Science
Carnegie Mellon University
This schedule is tentative and subject to change. Please check back often.
| Date | Lecture | Readings | Announcements |
|---|---|---|---|
Theory I: Fundamentals |
|||
| Mon, 28-Aug | Lecture 1
:
Overview of optimization [Notes] |
|
|
| Wed, 30-Aug | Lecture 2
:
Overview of optimization / Convex Sets [Notes] |
|
|
| Fri, 1-Sep |
|
|
|
| Mon, 4-Sep |
Labor Day |
|
|
| Wed, 6-Sep | Lecture 3
:
Convex sets [Notes] |
|
HW1 out (L1-L4)
|
| Fri, 8-Sep | Lecture 4
:
Convex functions [Notes] |
|
|
| Mon, 11-Sep |
Recitation: HW1 |
|
|
| Wed, 13-Sep | Lecture 5
:
Convex functions / Smoothness [Notes] |
|
|
Algorithms I: First-order methods |
|||
| Fri, 15-Sep | Lecture 6
:
Strong convexity / Optimality conditions [Notes] |
|
|
| Sun, 17-Sep |
|
|
HW1 due
|
| Mon, 18-Sep | Lecture 7
:
Gradient descent [Notes] |
|
|
| Wed, 20-Sep | Lecture 8
:
Convergence of gradient descent [Notes] |
|
Quiz 1 (in-class) HW2 out (L5-L8)
|
| Fri, 22-Sep |
Recitation: HW2 |
|
|
| Mon, 25-Sep | Lecture 9
:
Convergence of GD / Subgradients [Notes] |
|
|
| Wed, 27-Sep | Lecture 10
:
The subgradient method [Notes] |
|
|
| Fri, 29-Sep |
(No Recitation) |
|
|
| Sun, 1-Oct |
|
|
HW2 due
|
| Mon, 2-Oct | Lecture 11
:
Projected gradient descent [Notes] |
|
HW3 out (L9-L12)
|
| Wed, 4-Oct | Lecture 12
:
Stochastic gradient descent [Notes] |
|
(Quiz 2 in-class)
|
| Fri, 6-Oct |
Recitation: HW3 |
|
|
| Mon, 9-Oct | Lecture 13
:
Convergence of SGD [Notes] |
|
|
Theory II: Duality |
|||
| Wed, 11-Oct | Lecture 14
:
Duality in linear programs [Notes] |
|
(Quiz 3 in-class)
|
| Thu, 12-Oct |
|
|
HW3 due
|
| Fri, 13-Oct |
(No Recitation) |
|
|
| Mon, 16-Oct |
Fall break |
|
|
| Tue, 17-Oct |
|
|
|
| Wed, 18-Oct |
Fall break |
|
|
| Thu, 19-Oct |
|
|
|
| Fri, 20-Oct |
Fall break |
|
|
| Mon, 23-Oct | Lecture 15
:
Lagrangian duality [Notes] |
|
HW4 out (L13-L16)
|
Algorithms II: Second-order methods |
|||
| Wed, 25-Oct | Lecture 16
:
Newton's method / Log-barrier method [Notes] |
|
|
| Fri, 27-Oct |
Recitation: HW4 |
|
|
| Mon, 30-Oct | Lecture 17
:
Newton's method analysis [Notes] |
|
|
| Wed, 1-Nov | Lecture 18
:
Quasi-Newton methods / KKT conditions [Notes] |
|
(Quiz 4 in-class)
|
| Thu, 2-Nov |
|
|
|
| Fri, 3-Nov |
(No Recitation) |
|
HW4 due
|
| Mon, 6-Nov |
(Lecture cancelled) |
|
|
| Wed, 8-Nov |
In-class Exam |
|
Project description out
|
| Fri, 10-Nov |
(Discussions with instructor about project ideas) |
|
|
Algorithms III: Advanced methods |
|||
| Mon, 13-Nov | Lecture 19
:
Proximal gradient descent [Notes] |
|
Project team formation due by 2pm
|
| Wed, 15-Nov | Lecture 20
:
Momentum / Nesterov Acceleration [Notes] |
|
|
| Thu, 16-Nov |
|
|
Project proposal due
|
| Fri, 17-Nov |
|
|
|
| Sat, 18-Nov |
|
|
HW625 out
|
| Mon, 20-Nov | Lecture 21
:
Adaptive gradient methods / Mirror Descent / AdaGrad [Notes] |
|
(Quiz 5 in-class)
|
| Wed, 22-Nov |
Thanksgiving Holiday- No class |
|
|
| Thu, 23-Nov |
Thanksgiving Holiday- No class |
|
|
| Fri, 24-Nov |
Thanksgiving Holiday- No class |
|
|
| Mon, 27-Nov | Lecture 22
:
Adaptive gradient methods / RMSProp / Adam / Regret minimization [Notes] |
|
|
| Wed, 29-Nov |
(Lecture rescheduled to Friday) |
|
|
| Thu, 30-Nov |
|
|
Project checkpoint due
|
| Fri, 1-Dec | Lecture 23
:
Online gradient descent / Parallel and distributed SGD [Notes] |
|
|
Nonconvex optimization |
|||
| Mon, 4-Dec | Lecture 24
:
Parallel and distributed SGD / (Nearly)-convex optimization [Notes] |
|
HW625 due
|
| Wed, 6-Dec | Lecture 25
:
Theories of non-convex optimization / Special topics: Optimization and ML [Notes] |
|
|
| Fri, 8-Dec |
|
|
|
| Sun, 10-Dec |
|
|
Project poster due
|
| Tue, 12-Dec |
Project Final Presentations (details will be announced on Piazza) |
|
|
| Wed, 13-Dec |
|
|
Project final report due
|