10701, Fall 2016
School of Computer Science
Carnegie Mellon University
This schedule is tentative and subject to change. Please check back often.
The first lecture for 10701 is Wednesday, September 7th, 2016. That is, we start during the second week of classes.
Date  Lecture  Topics  Readings  Announcements 

Background 

Wed, 9/7/16  Lecture 1
(Eric) :
Intro to probability, MLE [Slides] [Annotated Slides] [Video] 
Course introduction, Basic probability, Maximum likelihood estimate 


Supervised Learning 

Mon, 9/12/16  Lecture 2
(Eric) :
Classification, kNN [Slides] [Annotated Slides] [Video] 
Optimal decision using Bayes rule, Types of classifiers, Effect of values of k on kNN classifiers, Probabilistic interpretation of kNN 

HW 1 Out

Wed, 9/14/16  Lecture 3
(Matt) :
Naive Bayes [Slides] [Video] 
Problems with estimating full joints, Advantages of Naive Bayes assumptions, Applications to discrete and continuous cases, Problems with Naive Bayes classifiers 


Mon, 9/19/16  Lecture 4
(Matt) :
Linear regression [Slides] [Video] 
Basic model, Solving linear regression, Error in linear regression, Advanced regression models 


Wed, 9/21/16  Lecture 5
(Matt) :
Logistic regression [Slides] [Video] 
Logistic regression vs. linear regression, Sigmoid funcion, MLE via gradient ascent, Regularization, Logistic regression for multiple classes 


Mon, 9/26/16  Lecture 6
(Eric) :
SVM [Slides] [Annotated Slides] [Video] 
Support vector machines, Primal and dual versions of SVM, Duality, KKT conditions  HW2 Out HW1 Due 

Wed, 9/28/16  Lecture 7
(Eric) :
Kernels & The Kernel Trick [Slides] [Annotated Slides] [Video] 
Kernel trick, [SVM Optimization e.g. Sequential Minimal Optimization (SMO)] 


Mon, 10/3/16  Lecture 8
(Eric) :
Ensemble learning  Boosting, Random Forests [Slides] [Annotated Slides] [Video] 
Combing weak learners, Bagging and random forest, AdaBoost, Algorithm and generalization bounds, Gradient boosting 

Proposal Due 
Theory 

Wed, 10/5/16  Lecture 9
(Eric) :
Learning theory [Slides] [Annotated Slides] [Video] 
Realizable vs agnostic, PAC learning in finite concept class, Sample complexity 


Mon, 10/10/16  Lecture 10
(Eric) :
VC dimension [Slides] [Annotated Slides] [Video] 
Sample complexity for infinite concept classes, VC dimension as a complexity measure, Structural risk minimization 

HW3 Out HW2 Due 
Wed, 10/12/16  Lecture 11
(Eric) :
Evaluating classifiers, Biasvariance decomposition [Slides] [Annotated Slides] [Video] 
Biasvariance decomposition, Structural risk minimization, Ways to avoid overfitting 


Supervised Learning 

Mon, 10/17/16  Lecture 12
(Matt) :
Perceptron, Neural networks [Slides] [Video] 
Perceptron, Multilayer Perceptron, Backpropagation 


Wed, 10/19/16  Lecture 13
(Matt) :
Deep Learning [Slides] [Video] 
"Deep" Learning, Convolutional Neural Networks, Layerwise Pretraining 


Unsupervised Learning 

Mon, 10/24/16  Lecture 14
(Matt) :
PCA and dimension reduction [Slides] [Video] 
Principal component analysis, Dimensionality reduction 

HW4 Out HW3 Due 
Wed, 10/26/16  Lecture 15
(Eric) :
Kmeans [Slides] [Video] 
Hierarchical clustering, Kmeans and Gaussian mixture models, Number of clusters 


Mon, 10/31/16  Lecture 16
(Eric) :
EM [Slides] [Annotated Slides] [Video] 
Expectation Maximization 


Wed, 11/2/16 
Midterm 



Probabilistic Modeling 

Mon, 11/7/16  Lecture 17
(Eric) :
Graphical models, Bayes nets [Slides] [Annotated Slides] [Video] 
Representation 
Midway Report Due 

Wed, 11/9/16  Lecture 18
(Eric) :
Inference and learning of graphical models [Slides] [Annotated Slides] [Video] 
Learning, Exact Inference 


Mon, 11/14/16  Lecture 19
(Matt) :
HMMs and CRFs [Slides] [Video] 
Directed vs. undirected, Undirected graphical models, Conditional random fields  HW5 Out HW4 Due 

Wed, 11/16/16  Lecture 20
(Matt) :
HMMs and CRFs (continued) [Slides] [Video] 



Mon, 11/21/16  Lecture 21
(Matt) :
Topic modeling and Approximate Inference [Slides] [Video] 
Advanced probabilistic modeling, Approximate inference, MCMC 


Wed, 11/23/16  Lecture (No Class: Thanksgiving)
:




Advanced Topics 

Mon, 11/28/16  Lecture 22
(Eric) :
Distributed ML [Slides] [Video] 

HW5 Due


Wed, 11/30/16  Lecture 23
(Matt) :
MDPs, Reinforcement learning [Slides] 
Markov decision processes, Value iteration, Policy iteration, QLearning 


Poster Session 

Fri, 12/2/16  Lecture 25
:
NSH 3305 
2:30  5:30 pm 


Project Report 

Fri, 12/9/16 
Project Report Due 

