Syllabus and (tentative) Course Schedule

Date Lecture Topics Readings and useful links
Module 1
Intro to Functional Approximation
Mon 9/12 Lecture 1: Overview.
Slides, (Annotated Slides)
Overview of Machine Learning
  • What is Machine Learning?
  • Elements of machine learning

Decision tree learning
  • Mitchell: Chap 1,3
  • Decision Tree Learning [Applet]
  • Leo Breiman (2001). Random Forests. Machine Learning Volume 45, Number 1.

Wed 9/14 Lecture 2: Nonparametric methods. Slides,
(Annotated Slides)
Non parametric learning methods
  • Univariate prediction without using a model: good or bad?
  • instance based "learning": the Bayes optimal classifier.
  • spectrum graph methods
Preliminaries: Learning linear separation functions
Mon 9/19 Lecture 3: Generative versus discriminative classifers.
Slides , (Annotated Slides)
  • Naive Bayes vs Logistic Regression
  • Design of an experiment
HW 1 out
Wed 9/21 Lecture 4: Linear regression and sparsity.
Slides (Annotated Slides)
  • Linear regression
  • Introduction to sparsity and structure.
Into the nonlinear world and theoretical foundations of supervised learning
Mon 9/26 Lecture 5: Neural Networks
Slides (Annotated Slides)
Neural networks and deep learning
Wed 9/28 Lecture 6: Computational Learning Theory
Slides (Annotated Slides)
Computational and Learning theory
  • PAC learning
  • VC dimensions
  • Structural risk minimization
Mon 10/3
Lecture 7: Overfitting and model selection
Slides (Annotated Slides)
Overfitting and model selection
HW 1 due, HW 2 and data out
Unsupervised learning: Clustering
Wed 10/5
Lecture 8: Clustering
Slides(Annotated Slides)
  • K-means
  • Distance metrics
Mon 10/10
Lecture 9: Expectation Maximization
Slides (Annotated Slides)
Probabilistic models for clustering: Expectation-maximization
Wed 10/12
Lecture 10: Infinite Mixture Models
Slides (Annotated Slides)
Infinite Clusters
  • Dirichlet processes
  • Introduction to sampling methods
Structured Inference: Graphical Models
Mon 10/17 Lecture 11: Hidden Markov Models
Slides (Annotated Slides)
Sequential Labeling: Hidden Markov Model
HW 2 due, HW 3 out, Project proposal due
Wed 10/19 Lecture 12: Conditional Random Fields Slides (Annotated Slides) Conditional Random Field: a discriminative HMM
Mon 10/24 Lecture 13: Bayesian Networks Slides (Annotated Slides) Bayesian Networks
  • A Pedigree of people
  • Exact inference algorithm
  • Bishop: Chap 8
Wed 10/26 Midterm Exam open book, open notes, no computers Will cover lectures upto 10/19
Mon 10/31 Lecture 14: Inference and Learning for Bayesian Networks Slides (Annotated Slides) Inference and Learning for Bayesian Networks HW 3 due
Wed 11/2 Lecture 15: Undirected Graphical Models and Approximate Inference Slides (Annotated Slides) Undirected Graphical Models and Approximate Inference HW 4 out
Alternative strategies of learning
Mon 11/7 Lecture 16: PCA versus Topic models
Slides (No annotated slides)
Subspace learning: nonprobabilistic vs probabilistic approaches
  • Principal component analysis
  • Topic models
  • David M. Blei, Andrew Y. Ng, Michael I. Jordan (2003). Latent Dirichlet Allocation. Journal of Machine Learning Research, Vol. 3 , pp. 993-1022.

Wed 11/9 Lecture 17: Support Vector Machines Slides (Annotated Slides) Support Vector Machines
Mon 11/14 Lecture 18: Structured Sparsity in Genetics (Lecturer: Seyoung Kim)
Structured sparsity in genetics Project mid-term report due
Wed 11/16 Lecture 19: Generative Latent Variable Models of Text (Lecturer: Jacob Eisenstein) Slides Social media modeling and analysis via latent space models HW 4 due, HW 5 out
Advanced Topics
Mon 11/21 Lecture 20: Advanced topics in maximum-margin learning
Slides (Annotated Slides)
  • The kernel trick
  • Maximum-entropy discrimination
  • Maximum-margin markov networks

Wed 11/23 No class for Thanksgiving Break
Mon 11/28 Lecture 21: Max-margin learning of graphical models
Slides (Annotated Slides)
  • Maximum-entropy discrimination
  • Maximum-margin markov networks
Wed 11/30 Lecture 22: Ensemble Methods: Boosting from weak learners
Slides (Annotated Slides)
Boosting: ensemble of weak learners
Mon 12/5 Lecture 23: Reinforcement Learning
Reinforcement Learning
  • What is machine learning?
  • Elements of machine learning
HW 5 due
Wed 12/7 No class! No class!
Thu 12/8 Poster session NSH atrium 2:30-6:30pm
Project final report due
Tue 12/13 Final Exam Doherty Hall 2210, 1:00-4:00pm
One A4 sheet of paper allowed, Closed book, CLosed notes.

© 2008 Eric Xing @ School of Computer Science, Carnegie Mellon University
[validate xhtml]