Tom Mitchell and Andrew W.
Center for Automated Learning and Discovery
School of Computer Science, Carnegie Mellon University
It is hard to imagine anything more fascinating than systems that automatically improve their own performance through experience. Machine learning deals with computer algorithms for learning from many types of experience, ranging from robots exploring their environments, to mining pre-existing databases, to actively exploring and mining the web. This course is designed to give PhD students a thorough grounding in the methodologies, technologies, mathematics and algorithms needed to do research in learning and data mining, or to apply learning or data mining techniques to a target problem.
The topics of the course draw from classical statistics, from machine learning, from data mining, from Bayesian statistics and from statistical algorithmics.
Students entering the class with a pre-existing working knowledge of probability, statistics and algorithms will be at an advantage, but the class has been designed so that anyone with a strong numerate background can catch up and fully participate.
YOU ARE ON THE WAIT LIST:
This class if now fully subscribed. You may want to consider
Class lectures: Tuesdays & Thursdays 10:30am-11:50am, Wean Hall 7500 starting on Tuesday September 13th, 2005
Review sessions: Thursdays 5-6pm, Location NSH 1305, starting on thursday September 15. TA's will cover material from lecture and the homeworks, and answer your questions. These review sessions are optional (but very helpful!).
topic and readings
|Thu Sep 8
lecture: warm-up review of some basic
and a Machine Learning algorithm
Tu Sep 13
Decision Tree learning
Maximum likelihood estimation, MAP estimation
|Th Sep 15
|Fast tour of useful concepts in probability
pdf ps.gz Corrections Solutions
Tu Sep 20
and Basis Functions
Tu Sep 27
Bayes classifier, MLE and MAP estimates
pdf train-1.txt test-1.txt plotGauss.m Solutions
Discriminative and Generative Models
|Th Sep 29
|Logistic regression, Generative and discriminative classifiers, maximizing conditional data likelihood, MLE and MAP estimates.
and instance-based learning
|Gaussian Mixture Models
everything up to this date. Open book,
notes. Closed computer.
Come to class by 10.30am promptly. You will then have 80 minutes to answer six mostly-short questions on material covered in the lectures and readings up to and including October 11th. We strongly advise you to practice using previous exams, so you know what to expect. try doing the previous exams first, and then look at the solutions. You will be allowed to look at your notes in class, but don't rely on this because you will run out of time unless you are sufficiently familiar with the material that you can just do the questions without needing to look up the techniques.
In addition, to help prepare, there will be a review at the recitation session at 5pm Thursday Oct 13th, and there will be another review on Monday Oct 17th, 6pm-7.30pm in NSH 1305.
Previous examinations for practice.
|Project proposals due
|Computational learning theory
Learning I: sample complexity, agnostic learning
| PAC Learning
II: VC dimension, SRM, Mistake bounds
|Margin based approaches
and optimization methods
| Bayes nets:
representation, conditional independence
|Bayes nets: inference, variable elimination, etc.
|Bayes nets: learning parameters and structure (fully observed data, and begin EM)
|EM and semi-supervised learning
| EM for Bayes
networks and Mixtures of Gaussians
|Hidden Markov Models: representation and learning
|Time series models
|Graphical Models: an overview of more advanced probabistic models that fall under a category called Graphical Models. This lecture defines and talks about specifric instances, such as Kalman filters, undirected graphs and Dynamic Bayesian Networks
|Final project reports due
|Mon Nov 21
|Project poster session: 4-6:30pm in the Newell-Simon Hall Atrium
|Project poster session
|Dimensionality Reduction: Feature selection, PCA, SVD, ICA, Fisher discriminant
|Advanced topic: Machine Learning and Text Analysis
Inference notes Solutions
|Markov decision processes: Predicting the results of decisions in an uncertain world.
learning: Learning policies to
maximize expected future rewards in an uncertain world.
|Scaling: Some of Andrew's favorite data structures and algorithms for tractable statistical machine learning.
19 8:30-11:30a.m at HH B103 and HH B131 (Hammerschlag Hall). No rescheduling possible.
open book, open notes, closed computer.
Course Website (this page):
Note to people outside CMU: Please feel free to reuse any of these course materials that you find of use in your own courses. We ask that you retain any copyright notices, and include written notice indicating the source of any materials you use.