INTRO TO MACHINE LEARNING

10-701, Fall 2018
WEH 7500, Mon & Wed 3:00PM - 4:20PM

Instructors Pradeep Ravikumar (pradeepr at cs dot cmu dot edu)
Ziv Bar-Joseph (zivbj at andrew dot cmu dot edu)
Teaching Assistants Daniel Bird (dpbird at andrew dot cmu dot edu)
Shubhranshu Shekhar (shubhras at andrew dot cmu dot edu)
Zirui Wang (ziruiw at andrew dot cmu dot edu)
Adithya Raghuraman (araghura at andrew dot cmu dot edu)
Jing Mao (jingmao at andrew dot cmu dot edu)
Umang Bhatt (usb at andrew dot cmu dot edu)
Sarah Mallepalle (smallepa at andrew dot cmu dot edu)
Yang Gao (yanggao at andrew dot cmu dot edu)
Chieh Lin (chiehl1 at andrew dot cmu dot edu)
Office Hours Pradeep Ravikumar: Tuesday: 2:00pm-3:00pm in GHC 8111
Ziv Bar-Joseph: Monday after class
Daniel Bird: Tuesday 9:00am-10:00am in GHC 8110
Shubhranshu Shekhar: Thursday 10:30am-11:30am in HBH 3037
Zirui Wang: Thursday 3:00pm-4:00pm at GHC 8th Floor Study Area
Adithya Raghuraman: Wednesday 10:00am-11:00am at GHC 8th Floor Dining Area
Jing Mao: Monday 10:00am-11:00am at GHC 8th Floor Dining Area
Umang Bhatt: Monday 11:00pm-12:00pm at GHC 8th Floor
Sarah Mallepalle: Friday 11:00am-12:00pm at GHC 5th Floor
Yang Gao: Tuesday 4:00pm-5:00pm in REH 244
Chieh Lin: Tuesday 2:00pm-3:00pm in GHC 8127

Grading 40% Homeworks, 15% Exam 1, 15% Exam 2, 25% Project, 5% Class Participation

Textbooks Lectures are intended to be self-contained. For supplementary readings, with each lecture, we will have pointers to either online reference materials, or chapters from the following books:
  • CB: Pattern Recognition and Machine Learning, Christopher Bishop.
  • KM: Machine Learning: A probabilistic perspective, Kevin Murphy.
  • HTF: The Elements of Statistical Learning: Data Mining, Inference and Prediction, Trevor Hastie, Robert Tibshirani, Jerome Friedman.
  • TM: Machine Learning, Tom Mitchell.

Course Details Syllabus. Piazza Discussion Board. Homeworks. Project. Livestream

Tentative Schedule
Date Inst. Topic Readings Notes
Module: Foundations
Aug 27 ZB Intro, Three Axes of ML: Data, Algorithms, Tasks, MLE (slides) KM Chap. 1
Aug 29 ZB Bayesian Estimation, MAP, Decision Theory, Model-free, Risk Minimization (slides, notes) KM Chap 2, 5, 6,
TM Chap 6
Sept 3 No Class, Labor Day
Sept 5 ZB Non-parametric Models: K nearest neighbors, Kernel regression (slides, notes) TM Chap. 8
HTF Chap. 6, 13
KM Chap. 14		 HW1 out
Sept 10 No Class, Jewish New Year
Module: Prediction, Parametric Methods
Sept 12 ZB Regression: Linear Regression (slides, notes (1), (2)) CB Chap. 3
Sept 17 PR Regularized, Polynomial, Logistic Regression (slides, notes) CB Chap. 4
Sept 19 PR Decision Trees (slides) TM Chap. 3
HTF Chap. 9		 HW 1 due/
HW2 out
Sept 24 PR Naive Bayes, Generative vs Discriminative (slides) CB Chap. 4
Sept 26 PR Neural Networks and Deep Learning (slides) CB Chap. 5
KM Chap. 28
Oct 1 PR Neural Networks and Deep Learning, I, II (slides, notes) CB Chap. 5
KM Chap. 28
Oct 3 ZB Support Vector Machines 1 (slides, notes) CB Chap. 6, 7 HW 2 due/
HW3 out
Oct 8 ZB Support Vector Machines 2 (slides, notes) CB Chap. 6, 7
Oct 10 ZB Boosting, Surrogate Losses, Ensemble Methods (slides, notes) HTF Chap 10
Module: Unsupervised Learning
Oct 15 ZB Clustering, Kmeans (slides, notes) HTF Chap. 14.1-14.3
Oct 17 ZB Clustering: Mixture of Gaussians, Expectation Maximization (slides, notes) CB Chap 9 HW 3 due
Oct 22 Midterm 17:00 - 19:00 Location: Rashid Auditorium
Module: Theoretical considerations
Oct 24 PR Generalization, Model Selection (slides) HTF Chap. 7 Project topic selection
Oct 29 PR Learning Theory: Statistical Guarantees for Empirical Risk Minimization (slides)
Module: Representation Learning
Oct 31 Guest Representation Learning: Feature Transformation, Random Features, PCA (slides) HTF Chap. 14.5 HW4 out
Nov 5 PR Representation Learning: PCA Continued, ICA (slides) HTF Chap. 14.7
Nov 7 PR Graphical Models: Representation (slides) KM Chap. 10, 19, 20 Project Proposal Submission
Module: Graphical and Sequence Models
Nov 12 PR Graphical Models: Inference (slides) KM Chap. 10, 19, 20
Nov 14 PR Graphical Models: Learning (slides) KM Chap. 10, 19, 20 HW 4 due
Nov 19 ZB Sequence Models: HMMs (slides) KM Chap. 17
Nov 21 No Class, Thanksgiving
Nov 26 BOTH Industry lecture
Nov 28 ZB Sequence Models: State Space Models, other time series models (slides1, slides2) KM Chap. 18
Module: Actions
Dec 3 ZB Reinforcement Learning (slides1) TM Chap 13
Dec 5 Exam 2
Dec 10 Final Project Presentations
Dec 13 Final Projects Due

Homeworks




    • Projects
  • Climate Data
  • Forensic Investigation
  • Music Generation
  • Single Cell Analysis
  • Social Circle Analysis
  • Speech Recognition
  • Identifying Duplicate Questions

    		•