Date 
Lecture 
Topics 
Readings and useful links 
Handouts 
Aug 27 (M) 
Course Overview

 Intro, admin
 Decision Tree Learning

 Mitchell Chapters 1, 2, 6.16.3
 Murphy Chapter 2
 Bishop Chapter 1, 2

Slides

Aug 29 (W) 
Learning Linear Separators

 Learning Linear Separators
 The Perceptron Algorithm
 Geometric Margins

 Chapter 9.1.1. and 9.1.2, ShalevShwartz and BenDavid
 Mitchell Chapters 4.1.2 and 4.4.1
 Bishop Chapter 4.1.7

Slides

Sept 3 (M)  No Class: Labour Day 
Sept 5 (W) 
Probability and Estimation, Naive Bayes 
 Estimating Probabilities from Data: MLE, MAP
 Naive Bayes, Conditional Independence


Slides
Slides

Sept 10 (M) 
Generative and Discriminative Classifiers, Logistic Regression, Naive Bayes 
 Naive Bayes, Text Classification and Bag of Words Representation
 Logistic Regression: Maximizing Conditional Likelihood


Slides
Slides

Sept 12 (W) 
Kernels 
 Kernels
 Kernelizing Algorithms
 Kernelizing Perceptron


Slides

Sept 17 (M) 
Generalization and Overfitting 

 Chapter 7, Mitchell
 Chapters 2,3,4 ShalevShwartz and BenDavid

Slides

Sept 19 (W) 
Generalization and Overfitting 
 Sample Complexity
 VC Dimension Based Bounds

 Chapter 7, Mitchell
 Chapters 2,3,4 ShalevShwartz and BenDavid

Slides

Sept 24 (M) 
Generalization and Overfitting 
 Sample Complexity
 Rademacher Based Bounds
 Model Selection


Slides

Sept 26 (W) 
Support Vector Machines 
 Primal and Dual Forms
 Kernalizing SVM


Slides

Oct 1 (M) 
Boosting 
 Weak learning, Strong learning, Adaboost


Slides

Oct 3 (W) 
Boosting, Model Selection 
 Margin based bounds for Boosting
 kfold cross validation
 Structural risk minimization

 Chapters 10 and 11 of ShalevShwartz and BenDavid

Slides
Slides
Slides

Oct 8 (M)  Midterm

Oct 10 (W) 
Linear Regression 
 Linear Regression
 Minimizing squared error and maximizing data likelihood


Slides

Oct 15 (M) 
Neural Networks 
 Neural Networks
 Backpropagation


Slides

Oct 17 (W) 
Deep Networks 
 Convolution
 Convolutional Neural Networks


Slides
Slides

Oct 22 (M) 
Active Learning 
 Active Learning
 Common heuristics, Sampling bias
 Safe Disagreement Based Active Learning Schemes


Slides

Oct 24 (W) 
SemiSupervised Learning 
 SemiSupervised Learning
 Transductive SVM
 Cotraining


Slides

Oct 29 (M) 
Graphical Models (Guest lecture by Matt Gormley) 
 Bayesian Networks
 Topic Models


Slides

Oct 31 (W) 
Graphical Models (Guest lecture by Matt Gormley) 
 Hidden Markov Models
 Conditional Random Fields


Slides

Nov 5 (M) 
Unsupervised Learning 
 Partitional Clustering
 Hierarchical Clustering


Slides

Nov 7 (W) 
Dimensionality Reduction 
 Principal Component Analysis
 Kernel Principal Component Analysis

 Bishop 12.1, 12.3
 Chapter 23 in ShalevShwartz and BenDavid book

Slides

Nov 12 (M) 
Online Learning 


Slides

Nov 14 (W) 
Deep Unsupervised Learning (Guest lecture by Russ) 
 Deep Unsupervised Learning


Slides

Nov 19 (M) 
Reinforcement Learning 
 Markov Decision Processes
 Value Iteration
 QLearning


Slides

Nov 21 (W)  No Class: Thanksgiving 
Nov 26 (M)  Project Presentations 
Nov 28 (W)  Project Presentations 
Dec 3 (M)  Final 
Dec 5 (W) 
Differential Privacy 


Slides
