Advanced Introduction to Machine Learning

10-715, Fall 2018

Carnegie Mellon University

Maria-Florina Balcan




This is a tentative schedule and is subject to change.

Date Lecture Topics Readings and useful links Handouts
Aug 27
(M)
Course Overview
  • Intro, admin
  • Decision Tree Learning
  • Mitchell Chapters 1, 2, 6.1-6.3
  • Murphy Chapter 2
  • Bishop Chapter 1, 2
Slides
Aug 29
(W)
Learning Linear Separators
  • Learning Linear Separators
  • The Perceptron Algorithm
  • Geometric Margins
  • Chapter 9.1.1. and 9.1.2, Shalev-Shwartz and Ben-David
  • Mitchell Chapters 4.1.2 and 4.4.1
  • Bishop Chapter 4.1.7
Slides
Sept 3
(M)
No Class: Labour Day
Sept 5
(W)
Probability and Estimation, Naive Bayes
  • Estimating Probabilities from Data: MLE, MAP
  • Naive Bayes, Conditional Independence
Slides
Slides
Sept 10
(M)
Generative and Discriminative Classifiers, Logistic Regression, Naive Bayes
  • Naive Bayes, Text Classification and Bag of Words Representation
  • Logistic Regression: Maximizing Conditional Likelihood
Slides
Slides
Sept 12
(W)
Kernels
  • Kernels
  • Kernelizing Algorithms
  • Kernelizing Perceptron
Slides
Sept 17
(M)
Generalization and Overfitting
  • Sample Complexity
  • Chapter 7, Mitchell
  • Chapters 2,3,4 Shalev-Shwartz and Ben-David
Slides
Sept 19
(W)
Generalization and Overfitting
  • Sample Complexity
  • VC Dimension Based Bounds
  • Chapter 7, Mitchell
  • Chapters 2,3,4 Shalev-Shwartz and Ben-David
Slides
Sept 24
(M)
Generalization and Overfitting
  • Sample Complexity
  • Rademacher Based Bounds
  • Model Selection
Slides
Sept 26
(W)
Support Vector Machines
  • Primal and Dual Forms
  • Kernalizing SVM
Slides
Oct 1
(M)
Boosting
  • Weak learning, Strong learning, Adaboost
Slides
Oct 3
(W)
Boosting, Model Selection
  • Margin based bounds for Boosting
  • k-fold cross validation
  • Structural risk minimization
  • Chapters 10 and 11 of Shalev-Shwartz and Ben-David
Slides
Slides
Slides
Oct 8
(M)
Midterm
Oct 10
(W)
Linear Regression
  • Linear Regression
  • Minimizing squared error and maximizing data likelihood
Slides
Oct 15
(M)
Neural Networks
  • Neural Networks
  • Backpropagation
Slides
Oct 17
(W)
Deep Networks
  • Convolution
  • Convolutional Neural Networks
Slides
Slides
Oct 22
(M)
Active Learning
  • Active Learning
  • Common heuristics, Sampling bias
  • Safe Disagreement Based Active Learning Schemes
Slides
Oct 24
(W)
Semi-Supervised Learning
  • Semi-Supervised Learning
  • Transductive SVM
  • Co-training
Slides
Oct 29
(M)
Graphical Models
(Guest lecture by Matt Gormley)
  • Bayesian Networks
  • Topic Models
  • TBD
Slides
Oct 31
(W)
Graphical Models
(Guest lecture by Matt Gormley)
  • Hidden Markov Models
  • Conditional Random Fields
  • TBD
Slides
Nov 5
(M)
Unsupervised Learning
  • Partitional Clustering
  • Hierarchical Clustering
Slides
Nov 7
(W)
Dimensionality Reduction
  • Principal Component Analysis
  • Kernel Principal Component Analysis
  • Bishop 12.1, 12.3
  • Chapter 23 in Shalev-Shwartz and Ben-David book
Slides
Nov 12
(M)
Online Learning
  • Online Learning
Slides
Nov 14
(W)
Deep Unsupervised Learning
(Guest lecture by Russ)
  • Deep Unsupervised Learning
  • TBD
Slides
Nov 19
(M)
Reinforcement Learning
  • Markov Decision Processes
  • Value Iteration
  • Q-Learning
Slides
Nov 21
(W)
No Class: Thanksgiving
Nov 26
(M)
Project Presentations
Nov 28
(W)
Project Presentations
Dec 3
(M)
Final
Dec 5
(W)
Differential Privacy
  • Differential Privacy
  • TBD
Slides