CONVEX OPTIMIZATION

10-725/36-725, Fall 2017
WH 7500, Mon & Wed 1:30 - 2:50 PM

Instructors Pradeep Ravikumar (pradeepr at cs dot cmu dot edu)
Aarti Singh (aarti at cs dot cmu dot edu)

Teaching Assistants Hao Gu (hg1 at andrew dot cmu dot edu)
Devendra Sachan (dsachan at andrew dot cmu dot edu)
Yifeng Tao (yifengt at andrew dot cmu dot edu)
Yichong Xu (yichongx at andrew dot cmu dot edu)
Hongyang Zhang (hongyanz at andrew dot cmu dot edu)

Office Hours Pradeep Ravikumar: GHC 8111, Mondays 3:00-4:00 PM
Aarti Singh: GHC 8207, Wednesdays 3:00-4:00 PM
Hao Gu: Citadel Teaching commons, GHC 5th floor, Tuesdays 4:00-5:00 PM
Devendra Sachan: LTI Open Space, 5th floor, Fridays 3:00-4:00 PM
Yifeng Tao: GHC 7405, Mondays 10:00-11:00 AM
Yichong Xu: GHC 8215, Tuesdays, 10:00-11:00 AM
Hongyang Zhang: GHC 8008, Wednesdays 9:00-10:00 AM

Grading 45% Homeworks, 5% Quizzes, 25% Little Tests, 25% Project

Textbooks Lectures are intended to be self-contained. For supplementary readings, with each lecture, we will have pointers to chapters from the following books:
  • BV: Convex Optimization, Stephen Boyd and Lieven Vandenberghe, (available online for free).
  • DB: Nonlinear Programming, Dimitri P. Bertsekas.
  • NW: Numerical Optimization, Jorge Nocedal and Stephen Wright.
  • YN: Introductory lectures on convex optimization: a basic course, Yurii Nesterov.


Course Details Syllabus. Piazza Discussion Board. Homeworks. Project.

Tentative Schedule
Date Topic Inst. Quiz Readings Notes
Module: Basics
Aug 28 Intro (slides) AS quiz
Aug 30 Convex Sets, Functions (slides, notes) AS quiz BV Chap. 2,3 HW 1a out (Sept 1)
Sep 06 Convex Opt. (slides, notes) AS quiz BV Chap. 4 HW 1b out (Sept 6)
Sep 11 Canonical Forms (slides) PR quiz BV Chap. 6,7,8 HW1a due (Sept 11)
Sep 13 Applications (slides ) PR quiz BV Chap. 6,7,8 HW1b due, HW 2 out (Sept 15)
Module: Unconstrained
Sep 18 Descent Alg., Line Search (slides ) PR quiz NW Chap. 3
Sep 20 Gradient Descent (slides ) PR quiz NW Chap. 3
Sep 25 Conjugate Grad. Desc.(slides ) AS quiz NW Chap. 5
CGThmProof (slides24-38)
Sep 27 Accel. Grad. Desc. (slides ) PR quiz YN Chap. 2 HW2 due, HW3 out(Sept 29)
Oct 02 Newton Methods (slides ) AS quiz NW Chap. 3
Convergence
Project proposal due (Oct 6)
Oct 04 Quasi Newton (slides ) AS quiz NW Chap. 3
Oct 11 Trust Region (slides ) PR quiz NW Chap. 4 HW3 due (Oct 13)
Module: Constrained: Simple Convex Sets
Oct 09 Proximal, Projected Grad. Desc. (slides, notes) AS quiz DB Chap. 2
Oct 16 Little Test I
Oct 18 Cond. Gradient/Frank Wolfe (slides, notes) AS quiz DB Chap. 2
Module: Constrained: Lagrange Methods
Oct 23 Lagrange Mult. Theory (slides) PR quiz DB Chap. 3 HW4 out (Oct 20)
Oct 25 Barrier Methods (slides) PR quiz DB Chap. 4
Oct 30 Augmented Lagrangian (slides) AS quiz DB Chap. 4
Nov 01 ADMM, Primal Dual Interior Point Methods (slides) (slides) AS quiz DB Chap. 4
Module: Constrained: Dual Methods
Nov 06 Duality Theory (slides) AS quiz DB Chap. 5 HW4 due, HW5 out (Nov 4)
Nov 08 Duality Theory (slides) AS quiz DB Chap. 5
Nov 13 Dual Ascent (slides) AS quiz DB Chap. 6
Nov 15 Coordinate Descent (slides) PR quiz NW Chap. 9
Nov 20 Subgradients, Cutting Plane Methods (slides) PR DB Chap. 6 HW5 due (Nov 18)
Module: Misc. Frameworks
Nov 22 No Class, Thanksgiving Break Project midterm due (Nov 22)
Nov 27 Decomposition Methods PR DB Chap. 6
Nov 29 Stochastic Optimization PR Papers
Dec 04 Non-smooth Optimization PR YN Chap. 3
Dec 06 Little Test II
Dec 13 Project Poster Pres. Project final report due (Dec 15)

Homeworks


  • HW 1 out. Prob 1-3 Due Sept 11, 5:00 pm, Prob 4 Due Sept 15 5:00 pm (for data files, please see Resources section in course Piazza page).
  • HW 2 out. Due Sept 29, 5:00 pm
  • HW 3 out. Due Oct 13, 5:00 pm
  • HW 4 out. Due Nov 4, 11:59 pm
  • HW 5 out. Due Nov 18, 11:59 pm
Project