INTRO TO MACHINE LEARNING

10-701, Fall 2018
WEH 7500, Mon & Wed 3:00PM - 4:20PM

Instructors Pradeep Ravikumar (pradeepr at cs dot cmu dot edu)
Ziv Bar-Joseph (zivbj at andrew dot cmu dot edu)

Teaching Assistants Daniel Bird (dpbird at andrew dot cmu dot edu)
Shubhranshu Shekhar (shubhras at andrew dot cmu dot edu)
Zirui Wang (ziruiw at andrew dot cmu dot edu)
Adithya Raghuraman (araghura at andrew dot cmu dot edu)
Jing Mao (jingmao at andrew dot cmu dot edu)
Umang Bhatt (usb at andrew dot cmu dot edu)
Sarah Mallepalle (smallepa at andrew dot cmu dot edu)
Yang Gao (yanggao at andrew dot cmu dot edu)
Akash Umakantha (aumakant at andrew dot cmu dot edu)

Office Hours Pradeep Ravikumar: TBD
Ziv Bar-Joseph: TBD
Daniel Bird: TBD
Shubhranshu Shekhar: TBD
Zirui Wang: TBD
Adithya Raghuraman: TBD
Jing Mao: TBD
Umang Bhatt: TBD
Sarah Mallepalle: TBD
Yang Gao: TBD
Akash Umakantha: TBD

Grading 50% Homeworks, 25% Midterm, 25% Project

Textbooks Lectures are intended to be self-contained. For supplementary readings, with each lecture, we will have pointers to either online reference materials, or chapters from the following books:
  • CB: Pattern Recognition and Machine Learning, Christopher Bishop.
  • KM: Machine Learning: A probabilistic perspective, Kevin Murphy.
  • HTF: The Elements of Statistical Learning: Data Mining, Inference and Prediction, Trevor Hastie, Robert Tibshirani, Jerome Friedman.
  • TM: Machine Learning, Tom Mitchell.

Course Details Syllabus. Piazza Discussion Board. Homeworks. Project.

Tentative Schedule
Date Inst. Topic Readings Notes
Module: Foundations
Aug 27 MV Intro (slides) KM Chap. 1
Aug 29 PR Prob. Models: Estimators, Guarantees, MLE (slides) KM Chap. 2, 6
Sept 3 No Class, Labor Day
Sept 5 MV Prob. Models: Bayesian Estimation, MAP (slides) TM Chap. 6,
KM Chap. 5
Sept 10 PR Model-free Methods, Decision Theory (slides) HTF Chap. 2 HW1 out
Module: Prediction, Parametric Methods
Sept 12 PR Regression: Linear Regression (slides) CB Chap. 3
Sept 17 MV Regularized, Polynomial, Logistic Regression (slides) CB Chap. 3, 4
Sept 19 PR Classification: Naive Bayes, Generative vs Discriminative (slides) CB Chap. 4
Sept 24 PR Classification: Support Vector Machines (slides) KM Chap. 14 HW 1 due/
(HW2out)
Sept 26 PR Classification: Boosting, Surrogate Losses (slides) HTF Chap. 10
Oct 1 MV Decision Trees (slides) TM Chap. 3,
HTF Chap. 9
Oct 3 PR Foundations: Generalization, Model Selection (slides) HTF Chap. 7
Oct 8 MV Neural Networks and Deep Learning (slides) CB Chap. 5,
KM Chap. 28
HW 2 due/
(HW3out)
Oct 10 MV Neural Networks and Deep Learning (slides) CB Chap. 5,
KM Chap. 28
Module: Non-Parametric Methods
Oct 15 PR Non-parametric Models: K nearest neighbors, kernel regression (slides) TM Chap. 8,
HTF Chap. 6, 13
Oct 17 PR Non-parametric Models: SVM, Lin Reg: primal + dual, Kernels, Kernel Trick (slides) CB Chap. 6, 7 HW 3 due (Mar 9)
Module: Unsupervised Learning
Oct 22 Midterm Review. Midterm Review (slides)
Oct 24 Midterm
Oct 29 PR Unsupervised Learning: Clustering, Kmeans (slides) HTF Chap. 14.1-14.3 (HW4out)
Oct 31 PR Unsupervised Learning: Clustering: Mixture of Gaussians, Expectation Maximization (slides) CB Chap. 9
Nov 5 PR Unsupervised Learning: Latent Variable Models (slides) CB Chap. 9
Nov 7 PR Unsupervised Learning: Graphical Models (slides) KM Chap. 10, 19, 20
Module: Sequence Models
Nov 12 MV Sequence Models: Hidden Markov Models (slides) KM Chap. 17 HW 4 due/
HW 5 out
Nov 14 MV Sequence Models: State Space Models, other time series models (slides) KM Chap. 18
Module: Representation Learning
Nov 19 TBD/PR Representation Learning: Feature Transformation, Random Features, PCA (slides) HTF Chap. 14.5
Nov 21 No Class, Thanksgiving
Nov 26 TBD/MV Representation Learning: PCA Contd, ICA (slides) HTF Chap. 14.7
Module: Reinforcement Learning
Nov 28 MV RL: MDPs, Value Iteration, Q Learning (slides) HW 5 due
Dec 3 MV RL: Q learning in non-det domains, Deep RL (slides)
Dec 5 PR Foundations: Statistical Guarantees for Empirical Risk Minimization (slides)
Dec 7 Final Project Presentations

Homeworks
  • HW 1 out. TBD
  • HW 2 out. TBD
  • HW 3 out. TBD
  • HW 4 out. TBD
  • Project Details