Date 
Lecture 
Topics 
Readings and useful links 
Handouts 
Sept 13
slides

Intro to ML
Decision Trees

 Machine learning examples
 Well defined machine learning problem
 Decision tree learning

Required:


Sept 15
slides

Decision Tree Learning
Review of Probability

 The big picture
 Overfitting
 Random variables, probabilities

Required:
 Bishop Ch.1 thru 1.2.3
 Bishop Ch.2 thru 2.2
Optional:


Sept 20
slides

Probability and Estimation

 Probability review
 Bayes rule
 MLE

Required:
 Bishop Ch.1 thru 1.2.3
 Bishop Ch.2 thru 2.2
Optional:


Sept 22
slides

Naive Bayes
MAP estimates

 Conditional independence
 Naive Bayes

Required:
Optional:


Sept 27
slides

Naive Bayes
MAP estimates

 MAP estimates, Conjugate priors
 Document classification

Required:
Optional:


Sept 29
slides

Gaussian Naive Bayes
Logistic Regression

 Gaussian Naive Bayes
 Brain image classification
 Logistic Regression
 Gradient ascent

Required:
Optional:


Oct 4
slides

Logistic Regression
Generative/Discriminative

 Logistic regression
 regularization and MAP estimation

Required:
 Bishop: Chapter 1.2.5
 Bishop: Chapter 3 through 3.2


Oct 6
slides

Linear regression

 linear regression
 polynomial regression
 biasvariance decomposition

Optional:


Oct 11
slides

Graphical Models 1

 Bayes nets
 Representing joint distributions with conditional independence assumptions
 Dseparation and conditional independence

Required:
Optional:


Oct 13
slides

Graphical Models 2


Required:
Optional:


Oct 18
slides

Graphical Models 3

 EM
 Mixture of Gaussians clustering
 Learning Bayes Net structure  Chow Liu

Required:
Optional:


Oct 20
slides

Computational Learning Theory 1


Optional:


Oct 25
PAC learning slides
Midterm Review slides

Computational Learning Theory 2

 PAC Learning
 VC Dimension
 Midterm review

Optional:


Oct 27

Midterm

Open book, Open notes, No computers



Nov 1
slides

Hidden Markov Models

 Markov models
 HMM's and Bayes Nets
 Other probabilistic time series models

Required:


Nov 3
slides

Neural Networks

 Nonlinear regression
 Backpropagation and gradient descent
 Learning hidden layer representations



Nov 8
slides

Learning Representations 1

 Feature Selection
 Principal Component Analysis (PCA)



Nov 10
slides

Learning Representations 2

 SVD
 ICA
 Laplacian Eigenmaps
 kmeans and spectral clustering



Nov 15
slides

Nonparametric methods

 Histogram and Kernel density estimation
 kNN Classifier
 Kernel Regression



Nov 17
slides

Support Vector Machines 1

 Maximizing margin
 SVM formulation
 Slack variables, hinge loss
 Multiclass SVM

 Bishop: Sec 7.1, Sec 4.1.1, 4.1.2, Appendix E


Nov 22
slides

Support Vector Machines 2

 Constrained optimization
 Dual SVM
 Kernel Trick
 Comparison with Kernel regression and logistic regression



Nov 29
slides

Boosting

 Combining weak classifiers
 Adaboost algorithm
 Comparison with logistic regression and bagging



Dec 1
slides

Semisupervised Learning

 Generative Methods
 Graphbased Methods
 Multiview Methods



Dec 6
slides

Active Learning

 Binary Bisection
 Uncertainty sampling
 QuerybyCommittee



Dec 8
slides

Review




Dec 16, 5:30  8:30 PM

Final Exam

Open book, Open notes, No computers


