Date 
Lecture 
Topics 
Readings and useful links 
Handouts 
Jan 12 
Intro to ML Decision Trees

 Machine learning examples
 Well defined machine learning problem
 Decision tree learning

Mitchell: Ch 3
Bishop: Ch 14.4
The
Discipline of Machine Learning

Slides
Video

Jan 14 
Decision Tree learning Review of Probability 
 The big picture
 Overfitting
 Random variables and probabilities

Mitchell: Ch 3
Andrew Moore's Basic Probability Tutorial

Slides
Annotated Slides
Video

Jan 21 
Probability and Estimation 

Mitchell: Estimating Probabilities
 Slides
Annotated Slides
Video

Jan 26 
Naive Bayes 
 Conditional Independence
 Naive Bayes: why and how

Mitchell:
Naive Bayes and Logistic Regression

Slides
Annotated Slides
Video

Jan 28 
Gaussian Naive Bayes 
 Gaussian Bayes classifiers
 Document Classification
 Brain image classification
 Form of decision surfaces

Mitchell:
Naive Bayes and Logistic Regression

Slides
Annotated Slides
Video

Feb 2 
Logistic Regression 
 Naive Bayes  the big picture
 Logistic Regression: Maximizing conditional likelihood
 Gradient ascent as a general learning/optimization method

Mitchell:
Naive Bayes and Logistic Regression

Slides
Annotated Slides
Video

Feb 4 
Linear Regression 
 Generative/Discriminative models
 Minimizing squared error and maximizing data likelihood
 Regularization
 Biasvariance decomposition


Slides
Annotated Slides
Video

Feb 9 
Learning Theory I 
 Distributional Learning
 PAC and Statistical Learning Theory
 Sample Complexity

Mitchell: Ch 7
Notes on Generalization Guarantees

Slides
Video

Feb 11 
Learning Theory II 
 Sample Complexity
 Shattering and VC Dimension
 Sauer's Lemma

Mitchell: Ch 7
Notes on Generalization Guarantees

Slides
Video

Feb 16 
Learning Theory III 
 Rademacher Complexity
 Overfitting and Regularization


Slides
Video

Feb 18 
Graphical Models I 
 Bayes Nets
 Representing joint distributions with conditional independence assumptions

Bishop chapter 8, through 8.2 
Slides
Annotated Slides
Video

Feb 23 
Graphical Models II 
 Inference
 Learning from fully observed data
 Learning from partially observed data


Annotated Slides
Video

Feb 25 
Graphical Models III 
 EM
 Semisupervised learning

Bishop Chapter 8 Mitchell Chapter 6 
Slides
Annotated Slides
Video

Mar 2  Exam #1 
Mar 4 
EM and Clustering 
 Mixture of Gaussian clustering
 Kmeans clustering

Bishop Chapter 8 Mitchell Chapter 6 
Slides
Annotated Slides
Video

Spring Break 
Mar 16 
Boosting 
 Weak vs Strong (PAC) Learning
 Boosting Accuracy
 Adaboost


Slides
Video

Mar 18 
Adaboost, Margins, Perceptron 
 Adaboost: Generalization Guarantees(naive and margins based).
 Geometric Margins and Perceptron

Notes on Perceptron 
Slides
Slides (PPT)
Video

Mar 23 
Kernels 
 Geometric Margins
 Kernels: Kernelizing a Learning Algorithm
 Kernelized Perceptron

Bishop 6.1 and 6.2 
Slides
Video

Mar 25 
SVM 
 Geometric Margins
 SVM: Primal and Dual Forms
 Kernelizing SVM
 Semisupervised Learning
 Semisupervised SVM

Notes on SVM by Andrew Ng

Slides
Video

Mar 30 
Semisupervised Learning 
 Transductive SVM
 Cotraining and Multiview Learning
 Graphbased Methods


Slides
Video

Apr 1 
Active Learning 
 Batch Active Learning
 Selective Sampling and Active Learning
 Sampling Bias


Slides
Video

Apr 6 
 Partitional Clustering
 Hierarchical Clustering

 kmeans, Lloyd's method, kmeans++
 Agglomerative Clustering


Slides
Video

Apr 8 
 Learning Representations
 Dimensionality Reduction

 Principal Component Analysis
 Kernel Principal Component Analysis


Slides
Video

Apr 13 
Never Ending Learning 


Slides
Video

Apr 15 
Neural Networks Deep Learning 

Mitchell, Chapter 4

Slides
Video

Apr 20 
Reinforcement Learning 
 Markov Decision Processes
 Value Iteration
 Qlearning


Slides
Video

Apr 22 
Deep Learning Differential Privacy Discussion on the Future of ML 


Slides (Privacy)
Slides (Deep Nets)
Video

Apr 27 
Course review 



Apr 29  Exam #2 