Instructors 
Pradeep Ravikumar (pradeepr at cs dot cmu dot edu)
Aarti Singh (aarti at cs dot cmu dot edu)

Teaching Assistants 
Hao Gu (hg1 at andrew dot cmu dot edu)
Devendra Sachan (dsachan at andrew dot cmu dot edu)
Yifeng Tao (yifengt at andrew dot cmu dot edu)
Yichong Xu (yichongx at andrew dot cmu dot edu)
Hongyang Zhang (hongyanz at andrew dot cmu dot edu)

Office Hours 
Pradeep Ravikumar: GHC 8111, Mondays 3:004:00 PM
Aarti Singh: GHC 8207, Wednesdays 3:004:00 PM
Hao Gu: Citadel Teaching commons, GHC 5th floor, Tuesdays 4:005:00 PM
Devendra Sachan: LTI Open Space, 5th floor, Fridays 3:004:00 PM
Yifeng Tao: GHC 7405, Mondays 10:0011:00 AM
Yichong Xu: GHC 8215, Tuesdays, 10:0011:00 AM
Hongyang Zhang: GHC 8008, Wednesdays 9:0010:00 AM

Grading 
45% Homeworks, 5% Quizzes, 25% Little Tests, 25% Project

Textbooks 
Lectures are intended to be selfcontained. For supplementary readings, with each lecture, we will have pointers to chapters from the following books:
 BV: Convex Optimization, Stephen Boyd and Lieven Vandenberghe,
(available online for free).
 DB: Nonlinear Programming, Dimitri P. Bertsekas.
 NW: Numerical Optimization, Jorge Nocedal and Stephen Wright.
 YN: Introductory lectures on convex optimization: a basic course, Yurii Nesterov.

Course Details 
Syllabus. Piazza Discussion Board. Homeworks. Project.

Tentative Schedule 
Date 
Topic 
Inst. 
Quiz 
Readings 
Notes 
Module: Basics 
Aug 28 
Intro (slides) 
AS 
quiz 


Aug 30 
Convex Sets, Functions (slides, notes) 
AS 
quiz 
BV Chap. 2,3 
HW 1a out (Sept 1) 
Sep 06 
Convex Opt. (slides, notes) 
AS 
quiz 
BV Chap. 4 
HW 1b out (Sept 6) 
Sep 11 
Canonical Forms (slides) 
PR 
quiz 
BV Chap. 6,7,8 
HW1a due (Sept 11) 
Sep 13 
Applications (slides ) 
PR 
quiz 
BV Chap. 6,7,8 
HW1b due, HW 2 out (Sept 15) 
Module: Unconstrained 
Sep 18 
Descent Alg., Line Search (slides ) 
PR 
quiz 
NW Chap. 3 

Sep 20 
Gradient Descent (slides ) 
PR 
quiz 
NW Chap. 3 

Sep 25 
Conjugate Grad. Desc.(slides ) 
AS 
quiz 
NW Chap. 5
CGThmProof (slides2438) 

Sep 27 
Accel. Grad. Desc. (slides ) 
PR 
quiz 
YN Chap. 2 
HW2 due, HW3 out(Sept 29) 
Oct 02 
Newton Methods (slides ) 
AS 
quiz 
NW Chap. 3 Convergence 
Project proposal due (Oct 6) 
Oct 04 
Quasi Newton (slides ) 
AS 
quiz 
NW Chap. 3 

Oct 11 
Trust Region (slides ) 
PR 
quiz 
NW Chap. 4 
HW3 due (Oct 13) 
Module: Constrained: Simple Convex Sets 
Oct 09 
Proximal, Projected Grad. Desc. (slides, notes) 
AS 
quiz 
DB Chap. 2 

Oct 16 
Little Test I 




Oct 18 
Cond. Gradient/Frank Wolfe (slides, notes) 
AS 
quiz 
DB Chap. 2 

Module: Constrained: Lagrange Methods 
Oct 23 
Lagrange Mult. Theory (slides) 
PR 
quiz 
DB Chap. 3 
HW4 out (Oct 20) 
Oct 25 
Barrier Methods (slides) 
PR 
quiz 
DB Chap. 4 

Oct 30 
Augmented Lagrangian (slides) 
AS 
quiz 
DB Chap. 4 

Nov 01 
ADMM, Primal Dual Interior Point Methods (slides) (slides) 
AS 
quiz 
DB Chap. 4 

Module: Constrained: Dual Methods 
Nov 06 
Duality Theory (slides) 
AS 
quiz 
DB Chap. 5 
HW4 due, HW5 out (Nov 4) 
Nov 08 
Duality Theory (slides) 
AS 
quiz 
DB Chap. 5 

Nov 13 
Dual Ascent (slides) 
AS 
quiz 
DB Chap. 6 

Nov 15 
Coordinate Descent (slides) 
PR 
quiz 
NW Chap. 9 

Nov 20 
Subgradients, Cutting Plane Methods (slides) 
PR 
quiz 
DB Chap. 6 
HW5 due (Nov 18) 
Module: Misc. Frameworks 
Nov 22 
No Class, Thanksgiving Break 



Project midterm due (Nov 22) 
Nov 27 
Discrete Optimization: Relaxations, Branch and Bound (slides) 
PR 
quiz 
DB Chap. 5 

Nov 29 
Stochastic Optimization (slides) 
PR 
quiz 
Papers 

Dec 04 
Nonsmooth Optimization (slides) 
PR 

YN Chap. 3 

Dec 06 
Little Test II 




Dec 13 
Project Poster Pres. 



Project final report due (Dec 15) 


Homeworks 
 HW 1 out. Prob 13 Due Sept 11, 5:00 pm, Prob 4 Due Sept 15 5:00 pm (for data files, please see Resources section in course Piazza page).
 HW 2 out. Due Sept 29, 5:00 pm
 HW 3 out. Due Oct 13, 5:00 pm
 HW 4 out. Due Nov 4, 11:59 pm
 HW 5 out. Due Nov 18, 11:59 pm

Project 
