Divider

Machine Learning, 10-701 and 15-781, 2004

Ziv Bar-Joseph and Andrew W. Moore

School of Computer Science, Carnegie Mellon University

Fall 2004 (Starting Tue Sep 14th)

Divider

It is hard to imagine anything more fascinating than automated systems that improve their own performance. The study of learning from data is commercially and scientifically important. This course is designed to give a graduate-level student a thorough grounding in the methodologies, technologies, mathematics and algorithms currently needed by people who do research in learning and data mining or who may need to apply learning or data mining techniques to a target problem.

The topics of the course draw from classical statistics, from machine learning, from data mining, from Bayesian statistics and from statistical algorithmics.

Students entering the class with a pre-existing working knowledge of probability, statistics and algorithms will be at an advantage, but the class has been designed so that anyone with a strong numerate background can catch up and fully participate.

Class lectures: Tuesdays & Thursdays 10:30am-11:50am, Wean Hall 5409 starting on Tuesday September 14th, 2004

Review sessions: Tuesdays 4:30pm-6:00pm Wean Hall 4623 starting on Tuesday September 14th, 2004 (details)

Instructors:

Teaching Assistants:

Textbook:

Course Website (this page):

Grading:

Policy on collaboration

Homeworks will be done individually: each student must hand in their own answers. It is acceptable, however, for students to collaborate in figuring out answers and helping each other solve the problems. We will be assuming that, as participants in a graduate course, you will be taking the responsibility to make sure you personally understand the solution to any work arising from such collaboration.

Policy on late homework:

Homework assignments

Projects

Lecture schedule (and online slides when available)

Date

Lecture Information

Instructor

Tuesday Sept 14

Intro to Probability and Statistics

Ziv Bar-Joseph

Thursday Sept 16

PAC Learning, VC Dimension

Tom Mitchell

Tuesday September 21

Density Estimation, Confidence Intervals

  • Lecture 3 (PDF)
  • Note: Max will be giving a Matlab Tutorial tonight (4:30 - 6:00) in WeH 4623.
Ziv Bar-Joseph

Thursday September 23

Cross Validation, Regression

Ziv Bar-Joseph

Tuesday Sept 28

Maximum Likelihood Estimation, Estimator Bias

Ziv Bar-Joseph

Thursday Sept 30

Mixture Models, Expectation-Maximization

Carlos Guestrin

Tuesday Oct 5

Logistic Regression and Regularization

Ziv Bar-Joseph

Thursday Oct 7

K-Means and Hierarchical Clustering

  • Andrew's K-Means slides
  • For supplementary information, please see "Pattern Classification" by Duda, Hart, and Stork.
Andrew Moore

Tuesday Oct 12

Hierarchical and Spectral Clustering

Ziv Bar-Joseph

Thursday Oct 14

Generative vs. Discriminative Models, Bayes Classifiers

Ziv Bar-Joseph

Tuesday Oct 19

Decision Trees and Information Gain

Andrew Moore

Thursday Oct 21

Decision Trees (cont'd), Neural Networks

Andrew Moore

Tuesday Oct 26

Neural Networks (cont'd)

Andrew Moore

Thursday Oct 28

Instance-based Learning

Andrew Moore

Tuesday Nov 2

Midterm Exam

Thursday Nov 4

Support Vector Machines

Andrew Moore

Tuesday Nov 9

Support Vector Machines, Kernels

Andrew Moore

Thursday Nov 11

Learning from Labeled and Unlabeled Data

Ziv Bar-Joseph

Tuesday Nov 16

Markov Models and Hidden Markov Models

Ziv Bar-Joseph

Thursday Nov 18

Hidden Markov Models (cont'd)

Ziv Bar-Joseph

Tuesday Nov 23

Markov Decision Processes and Reinforcement Learning

Andrew Moore

Thursday Nov 25

No class

Tuesday Nov 30

Computational Biology

Ziv Bar-Joseph

Thursday Dec 2

Bayesian Networks

Drew Bagnell

Tuesday Dec 7

Bayesian Networks (cont'd)

Andrew Moore

Thursday Dec 9

Active Learning

Andrew Moore

Review sessions

Date
Time
Place
Instructor
Topic
Tue Sep. 14
4:30pm ~ 6:00pm
WeH 4623
Ziv Bar-Joseph
Introduction to Basic Probability
Tue Sep. 21
4:30pm ~ 6:00pm
WeH 4623
Max Likhachev
Matlab Tutorial
Tue Sep. 28
4:30pm ~ 6:00pm
WeH 4623
Yanjun Qi & Max Likhachev
Review for homework 1
Tue Oct. 5
4:30pm ~ 6:00pm
WeH 4623
Yanjun Qi & Max Likhachev
Review for homework 2
Tue Oct. 12
4:30pm ~ 6:00pm
WeH 4623
Yanjun Qi & Max Likhachev & Dave Ferguson
Review for homework 2
Tue Oct. 19
4:30pm ~ 6:00pm
WeH 4623
Max Likhachev & Dave Ferguson & Yanjun Qi
Homework 2 Solutions
Review for Homework 3
Tue Oct 26
4:30pm ~ 6:00pm
WeH 4623
Max Likhachev & Yanjun Qi
Homework 2 Solutions
Review for Homework 3
Spectral Clustering Notes
Thur Oct 28
5:00pm
NSH 1305
Ziv Bar-Joseph
Midterm Review (lectures 1 - 9)
Mon Nov 1
5:00pm
NSH 3002
Andrew Moore
Midterm Review (lectures 10 - 13)
Thur Dec 9
4:30 - 7:00 pm
NSH 1305
Ziv Bar-Joseph
Final Exam Review (session 1)
Fri Dec 10
5:00 - 6:30 pm
WeH 4615
Andrew Moore
Final Exam Review (session 2)
Sat Dec 11
5:00 - 6:30 pm
NSH 1305
Andrew Moore
Final Exam Review (session 3)

Exam Schedule

Additional Resources

Here are some example questions here for studying for the midterm/final. Note that these are exams from earlier years, and contain some topics that will not appear in this year's final. And some topics will appear this year that do not appear in the following examples.

Note to people outside CMU

Feel free to use the slides and materials available online here. Please email the instructors with any corrections or improvements. Additional slides and software are available at the Machine Learning textbook homepage and at Andrew Moore's tutorials page.