Ziv BarJoseph and Andrew W. Moore
School of Computer Science, Carnegie Mellon University
It is hard to imagine anything more fascinating than automated systems that improve their own performance. The study of learning from data is commercially and scientifically important. This course is designed to give a graduatelevel student a thorough grounding in the methodologies, technologies, mathematics and algorithms currently needed by people who do research in learning and data mining or who may need to apply learning or data mining techniques to a target problem.
The topics of the course draw from classical statistics, from machine learning, from data mining, from Bayesian statistics and from statistical algorithmics.
Students entering the class with a preexisting working knowledge of probability, statistics and algorithms will be at an advantage, but the class has been designed so that anyone with a strong numerate background can catch up and fully participate.
Class lectures: Tuesdays & Thursdays 10:30am11:50am, Wean Hall 5409 starting on Tuesday September 14th, 2004
Review sessions: Tuesdays 4:30pm6:00pm Wean Hall 4623 starting on Tuesday September 14th, 2004 (details)
Instructors:
Textbook:
Course Website (this page):
Grading:
Policy on collaboration
Homeworks will be done individually: each student must hand in their own answers. It is acceptable, however, for students to collaborate in figuring out answers and helping each other solve the problems. We will be assuming that, as participants in a graduate course, you will be taking the responsibility to make sure you personally understand the solution to any work arising from such collaboration.
Policy on late homework:
Date 
Lecture Information 
Instructor

Tuesday Sept 14 
Intro to Probability and Statistics 
Ziv BarJoseph

Thursday Sept 16 
PAC Learning, VC Dimension

Tom Mitchell

Tuesday September 21 
Density Estimation, Confidence Intervals

Ziv BarJoseph

Thursday September 23 
Cross Validation, Regression 
Ziv BarJoseph

Tuesday Sept 28 
Maximum Likelihood Estimation, Estimator Bias 
Ziv BarJoseph

Thursday Sept 30 
Mixture Models, ExpectationMaximization

Carlos Guestrin

Tuesday Oct 5 
Logistic Regression and Regularization 
Ziv BarJoseph

Thursday Oct 7 
KMeans and Hierarchical Clustering

Andrew Moore

Tuesday Oct 12 
Hierarchical and Spectral Clustering 
Ziv BarJoseph

Thursday Oct 14 
Generative vs. Discriminative Models, Bayes Classifiers

Ziv BarJoseph

Tuesday Oct 19 
Decision Trees and Information Gain

Andrew Moore

Thursday Oct 21 
Decision Trees (cont'd), Neural Networks

Andrew Moore

Tuesday Oct 26 
Neural Networks (cont'd) 
Andrew Moore

Thursday Oct 28 
Instancebased Learning

Andrew Moore

Tuesday Nov 2 
Midterm Exam 

Thursday Nov 4 
Support Vector Machines 
Andrew Moore

Tuesday Nov 9 
Support Vector Machines, Kernels 
Andrew Moore

Thursday Nov 11 
Learning from Labeled and Unlabeled Data 
Ziv BarJoseph

Tuesday Nov 16 
Markov Models and Hidden Markov Models

Ziv BarJoseph

Thursday Nov 18 
Hidden Markov Models (cont'd) 
Ziv BarJoseph

Tuesday Nov 23 
Markov Decision Processes and Reinforcement Learning 
Andrew Moore

Thursday Nov 25 
No class 

Tuesday Nov 30 
Computational Biology 
Ziv BarJoseph

Thursday Dec 2 
Bayesian Networks 
Drew Bagnell

Tuesday Dec 7 
Bayesian Networks (cont'd) 
Andrew Moore

Thursday Dec 9 
Active Learning 
Andrew Moore

Date

Time

Place

Instructor

Topic

Tue Sep. 14

4:30pm ~ 6:00pm

WeH 4623

Ziv BarJoseph

Introduction
to Basic Probability

Tue Sep. 21

4:30pm ~ 6:00pm

WeH 4623

Max Likhachev

Matlab Tutorial

Tue Sep. 28

4:30pm ~ 6:00pm

WeH 4623

Yanjun Qi & Max Likhachev

Review for homework 1

Tue Oct. 5

4:30pm ~ 6:00pm

WeH 4623

Yanjun Qi & Max Likhachev

Review for homework 2

Tue Oct. 12

4:30pm ~ 6:00pm

WeH 4623

Yanjun Qi & Max Likhachev & Dave Ferguson

Review for homework 2

Tue Oct. 19

4:30pm ~ 6:00pm

WeH 4623

Max Likhachev & Dave Ferguson & Yanjun Qi

Homework 2 Solutions
Review for Homework 3

Tue Oct 26

4:30pm ~ 6:00pm

WeH 4623

Max Likhachev & Yanjun Qi

Homework 2 Solutions
Review for Homework 3

Thur Oct 28

5:00pm

NSH 1305

Ziv BarJoseph

Midterm Review (lectures 1  9)

Mon Nov 1

5:00pm

NSH 3002

Andrew Moore

Midterm Review (lectures 10  13)

Thur Dec 9

4:30  7:00 pm

NSH 1305

Ziv BarJoseph

Final Exam Review (session 1)

Fri Dec 10

5:00  6:30 pm

WeH 4615

Andrew Moore

Final Exam Review (session 2)

Sat Dec 11

5:00  6:30 pm

NSH 1305

Andrew Moore

Final Exam Review (session 3)

Here are some example questions here for studying for the midterm/final. Note that these are exams from earlier years, and contain some topics that will not appear in this year's final. And some topics will appear this year that do not appear in the following examples.
Feel free to use the slides and materials available online here. Please email the instructors with any corrections or improvements. Additional slides and software are available at the Machine Learning textbook homepage and at Andrew Moore's tutorials page.