Machine Learning

10-601, Spring 2015

Carnegie Mellon University

Tom Mitchell and Maria-Florina Balcan




This is a tentative schedule and is subject to change.
Please note that Youtube takes some time to process videos before they become available.

Date Lecture Topics Readings and useful links Handouts
Jan 12 Intro to ML
Decision Trees
  • Machine learning examples
  • Well defined machine learning problem
  • Decision tree learning
Mitchell: Ch 3
Bishop: Ch 14.4
The Discipline of Machine Learning
Slides
Video
Jan 14 Decision Tree learning
Review of Probability
  • The big picture
  • Overfitting
  • Random variables and probabilities
Mitchell: Ch 3
Andrew Moore's Basic Probability Tutorial
Slides
Annotated Slides
Video
Jan 21 Probability and Estimation
  • Bayes rule
  • MLE
  • MAP
Mitchell: Estimating Probabilities Slides
Annotated Slides
Video
Jan 26 Naive Bayes
  • Conditional Independence
  • Naive Bayes: why and how
Mitchell: Naive Bayes and Logistic Regression Slides
Annotated Slides
Video
Jan 28 Gaussian Naive Bayes
  • Gaussian Bayes classifiers
  • Document Classification
  • Brain image classification
  • Form of decision surfaces
Mitchell: Naive Bayes and Logistic Regression Slides
Annotated Slides
Video
Feb 2 Logistic Regression
  • Naive Bayes - the big picture
  • Logistic Regression: Maximizing conditional likelihood
  • Gradient ascent as a general learning/optimization method
Mitchell: Naive Bayes and Logistic Regression Slides
Annotated Slides
Video
Feb 4 Linear Regression
  • Generative/Discriminative models
  • Minimizing squared error and maximizing data likelihood
  • Regularization
  • Bias-variance decomposition
  Slides
Annotated Slides
Video
Feb 9 Learning Theory I
  • Distributional Learning
  • PAC and Statistical Learning Theory
  • Sample Complexity
Mitchell: Ch 7
Notes on Generalization Guarantees
Slides
Video
Feb 11 Learning Theory II
  • Sample Complexity
  • Shattering and VC Dimension
  • Sauer's Lemma
Mitchell: Ch 7
Notes on Generalization Guarantees
Slides
Video
Feb 16 Learning Theory III
  • Rademacher Complexity
  • Overfitting and Regularization
  Slides
Video
Feb 18 Graphical Models I
  • Bayes Nets
  • Representing joint distributions with conditional independence assumptions
Bishop chapter 8, through 8.2 Slides
Annotated Slides
Video
Feb 23 Graphical Models II
  • Inference
  • Learning from fully observed data
  • Learning from partially observed data
  Annotated Slides
Video
Feb 25 Graphical Models III
  • EM
  • Semi-supervised learning
Bishop Chapter 8
Mitchell Chapter 6
Slides
Annotated Slides
Video
Mar 2Exam #1
Mar 4 EM and Clustering
  • Mixture of Gaussian clustering
  • K-means clustering
Bishop Chapter 8
Mitchell Chapter 6
Slides
Annotated Slides
Video
Spring Break
Mar 16 Boosting
  • Weak vs Strong (PAC) Learning
  • Boosting Accuracy
  • Adaboost
Slides
Video
Mar 18 Adaboost, Margins, Perceptron
  • Adaboost: Generalization Guarantees(naive and margins based).
  • Geometric Margins and Perceptron
Notes on Perceptron Slides
Slides (PPT)
Video
Mar 23 Kernels
  • Geometric Margins
  • Kernels: Kernelizing a Learning Algorithm
  • Kernelized Perceptron
Bishop 6.1 and 6.2 Slides
Video
Mar 25 SVM
  • Geometric Margins
  • SVM: Primal and Dual Forms
  • Kernelizing SVM
  • Semi-supervised Learning
  • Semi-supervised SVM
Notes on SVM by Andrew Ng Slides
Video
Mar 30 Semi-supervised Learning
  • Transductive SVM
  • Co-training and Multi-view Learning
  • Graph-based Methods
Slides
Video
Apr 1 Active Learning
  • Batch Active Learning
  • Selective Sampling and Active Learning
  • Sampling Bias
Slides
Video
Apr 6
  • Partitional Clustering
  • Hierarchical Clustering
  • k-means, Lloyd's method, k-means++
  • Agglomerative Clustering
Slides
Video
Apr 8
  • Learning Representations
  • Dimensionality Reduction
  • Principal Component Analysis
  • Kernel Principal Component Analysis
    Bishop 12.1, 12.3
Slides
Video
Apr 13 Never Ending Learning     Slides
Video
Apr 15 Neural Networks
Deep Learning
  Mitchell, Chapter 4 Slides
Video
Apr 20 Reinforcement Learning
  • Markov Decision Processes
  • Value Iteration
  • Q-learning
Slides
Video
Apr 22 Deep Learning
Differential Privacy
Discussion on the Future of ML
    Slides (Privacy)
Slides (Deep Nets)
Video
Apr 27 Course review      
Apr 29Exam #2