Machine Learning

10-701/15-781, Spring 2014

Barnabas PoczosAarti Singh

Home People Lectures Recitations Homeworks Project Previous material Table of algorithms
Date Lecture Lecturer Slides Useful links HWs
January 14 Tuesday Intro Aarti Slides
January 16 Thursday MLE and MAP estimation Barnabas MLE & MAP Slides (part1)
MLE & MAP Slides (part2)
Tail Bounds
January 21 Tuesday MLE & MAP, Naive Bayes Barnabas Naive Bayes
January 23 Thursday Naive Bayes, Linear regression Barnabas, Aarti Linear Regression
January 28 Tuesday Linear regression Aarti
January 30 Thursday Polynomial regression, Logistic regression Aarti Logistic Regression
February 4 Tuesday Logistic Regression, Support Vector Machine Aarti, Barnabas Support Vector Machines,
SVM (Annotated)
Kernel Methods HW1
HW1 handout
February 6 Thursday Support Vector Machines Barnabas Convex Optimization
February 11 Tuesday Support Vector Machines Barnabas Project proposal
February 13 Thursday Support Vector Machines, kernel density estimation Barnabas, Aarti Nonparametric Methods
February 18 Tuesday k-NN, kernel regression Aarti
February 20 Thursday Model selection, cross-validation Aarti Model Selection HW1 due
February 25 Tuesday k-means clustering, MoG, Expectation-Maximization Barnabas Expectation-Maximization,
EM (Annotated)
Max Welling's classnotes
February 27 Thursday Expectation-Maximization Barnabas Tom Minka's notes
March 4 Tuesday Hidden Markov Models Barnabas Hidden Markov Models
March 6 Thursday Hidden Markov Models, Decision Trees Barnabas, Aarti Decision Trees
March 11 Tuesday No Class - spring break HW2 due (March 10)
March 13 Thursday No Class - spring break
March 18 Tuesday Decision Trees, Graphical Models Aarti Graphical Models (I) HW3
March 20 Thursday Graphical Models Aarti Graphical Models (II)
March 25 Tuesday Graphical Models Aarti Graphical Models (III) Project Midterm (Mar 26)
March 27 Thursday Principal Component Analysis Barnabas PCA
April 1 Tuesday Independent Component Analysis Barnabas ICA
April 3 Thursday Boosting Aarti Boosting HW3 due
April 8 Tuesday Midterm
April 10 Thursday No Class - spring carnival
April 15 Tuesday Markov Chain Monte Carlo methods Barnabas MCMC
April 17 Thursday Learning Theory (I) Aarti PACBounds HW4 out
April 22 Tuesday Learning Theory (II) Aarti VCBounds
April 24 Thursday Neural Networks Aarti Neural Networks
April 29 Tuesday Deep learning Barnabas Deep Learning HW4 Due (Apr 27)
May 1 Thursday No lecture (Poster presentations) Project Demo (NSH Atrium, 2.30pm - 5.30pm)


Bishop: Sec 2.1, Appendix B
Mithcell: Ch 1

Andrew Moore's Basic Probability Tutorial
Bishop: Sec 2.2, 2.3 (up to 2.3.6)

Mitchell's Chapter Draft

Mitchell's Chapter Draft
Bishop: Sec 4.1-4.3
On Discriminative and Generative Classifiers, Ng and Jordan, NIPS, 2001 (pdf)
On gradient descent and Newton's method: Boyd's slides and Chapter 9 of Convex Optimization.

Least Squares Applet
Tutorial on regression by Andrew Moore
Bishop: Sec 3.1

Bishop: Sec 2.5, 6.3
Mitchell: Ch 8
Tutorial on Instance-based Learning by Andrew Moore

Bishop: Sec 1.3, 3.1.4
Hastie: Ch 7 (recommended)
A study of CV and Bootstrap (optional)
MDL website (optional)
Model Selection and MDL principle paper by M. Hansen and B. Yu (optional)

Mitchell: Ch 3
Decision Tree Applet

Bishop: Sec 14.3
Boosting homepage
Schapire: Boosting Tutorial, Video
Adaboost Applet

Bishop: Sec 7.1, Sec 4.1.1, 4.1.2,
Appendix E
Stephen Boyd's book: Ch 5 (optional)

Bishop: Sec 6.1, 6.2
Tutorials on SVMs and Kernels
Additional resource: SVM website

Bishop: Sec 9.1

Bishop: Ch 9

Mitchell: Ch 7

Bishop: Ch 13
HMM and EM Tutorial

Bishop: Ch 8
Graphical Models tutorial by M. Jordan
Intro to Graphical Models by K. Murphy

Belkin-Niyogi Paper on Laplacian Emaps
Spectral Clustering tutorial by Ulrike von Luxburg
Spectral Clustering demo

Derivation of Backpropagation (pdf)