Machine Learning

10-601, Fall 2011

Carnegie Mellon University

Tom Mitchell, Aarti Singh


Date Lecture Topics Readings and useful links Handouts
Sept 13

slides

Intro to ML
Decision Trees
  • Machine learning examples
  • Well defined machine learning problem
  • Decision tree learning
Required:  
Sept 15 slides Decision Tree Learning
Review of Probability
  • The big picture
  • Overfitting
  • Random variables, probabilities
Required:
  • Bishop Ch.1 thru 1.2.3
  • Bishop Ch.2 thru 2.2
Optional:
 
Sept 20 slides Probability and Estimation
  • Probability review
  • Bayes rule
  • MLE
Required:
  • Bishop Ch.1 thru 1.2.3
  • Bishop Ch.2 thru 2.2
Optional:
 
Sept 22 slides Naive Bayes
MAP estimates
  • Conditional independence
  • Naive Bayes
Required: Optional:  
Sept 27 slides Naive Bayes
MAP estimates
  • MAP estimates, Conjugate priors
  • Document classification
Required: Optional:  
Sept 29 slides Gaussian Naive Bayes

Logistic Regression
  • Gaussian Naive Bayes
  • Brain image classification
  • Logistic Regression
  • Gradient ascent
Required: Optional:  
Oct 4 slides Logistic Regression
Generative/Discriminative
  • Logistic regression
  • regularization and MAP estimation
Required:
  • Bishop: Chapter 1.2.5
  • Bishop: Chapter 3 through 3.2
 
Oct 6 slides Linear regression
  • linear regression
  • polynomial regression
  • bias-variance decomposition
Optional:  
Oct 11 slides Graphical Models 1
  • Bayes nets
  • Representing joint distributions with conditional independence assumptions
  • D-separation and conditional independence
Required: Optional:  
Oct 13 slides Graphical Models 2
  • D-separation
  • Inference
Required: Optional:    
Oct 18 slides Graphical Models 3
  • EM
  • Mixture of Gaussians clustering
  • Learning Bayes Net structure - Chow Liu
Required: Optional:    
Oct 20 slides Computational Learning Theory 1
  • PAC Learning
Optional:  
Oct 25
PAC learning slides
Midterm Review slides
Computational Learning Theory 2
  • PAC Learning
  • VC Dimension
  • Midterm review
Optional:    
Oct 27 Midterm Open book, Open notes, No computers  
Nov 1
slides
Hidden Markov Models
  • Markov models
  • HMM's and Bayes Nets
  • Other probabilistic time series models
Required:  
Nov 3
slides
Neural Networks
  • Non-linear regression
  • Backpropagation and gradient descent
  • Learning hidden layer representations
 
Nov 8
slides
Learning Representations 1
  • Feature Selection
  • Principal Component Analysis (PCA)
 
Nov 10
slides
Learning Representations 2
  • SVD
  • ICA
  • Laplacian Eigenmaps
  • k-means and spectral clustering
 
Nov 15
slides
Nonparametric methods
  • Histogram and Kernel density estimation
  • k-NN Classifier
  • Kernel Regression
 
Nov 17
slides
Support Vector Machines 1
  • Maximizing margin
  • SVM formulation
  • Slack variables, hinge loss
  • Multi-class SVM
  • Bishop: Sec 7.1, Sec 4.1.1, 4.1.2, Appendix E
 
Nov 22
slides
Support Vector Machines 2
  • Constrained optimization
  • Dual SVM
  • Kernel Trick
  • Comparison with Kernel regression and logistic regression
 
Nov 29
slides
Boosting
  • Combining weak classifiers
  • Adaboost algorithm
  • Comparison with logistic regression and bagging
 
Dec 1
slides
Semi-supervised Learning
  • Generative Methods
  • Graph-based Methods
  • Multi-view Methods
 
Dec 6
slides
Active Learning
  • Binary Bisection
  • Uncertainty sampling
  • Query-by-Committee
 
Dec 8
slides
Review      
Dec 16, 5:30 - 8:30 PM Final Exam Open book, Open notes, No computers